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ABSTRACT 

Inert  knowledge  is  typically  produced  in  learning  environments  that  simplify 
content  and  context.  Several  theories  and  methods  exist  to  establish  learning 
environments  that  overcome  the  negative  effects  of  inert  knowledge.  This  study 
examined  the  effects  of  combining  three  instructional  approaches  on  knowledge 
acquisition  by  advanced  learners  in  a  computer-based  learning  environment. 

This  study  used  a  2  X  2  experimental  design.  The  two  independent  variables 
were  program  version  and  method  of  instruction.  Two  qualitatively  different 
computer  programs  were  used:  a  base  program  developed  according  to  cognitive 
flexibility  theory  (CFT)  and  a  generative  program  containing  embedded  generative 
learning  strategies  (explanation  and  summary  cues).  Students  completed  these 
programs  either  individually  or  collaboratively. 

One  hundred  and  thirty-two  second-year  medical  students  participated  in  the 
study.  The  two  treatment  variables  were  randomly  assigned  to  the  students.  Students 
received  computer  diskettes  that  permitted  access  to  the  computer  module,  "Handling 
Transfusion  Hazards,"  and  were  given  instructions  on  how,  when,  and  where  to 
complete  the  program.  The  computer  program  consisted  of  an  orientation  section,  a 
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pretest,  a  learning  phase,  and  a  posttest.  The  pretest  and  posttest  consisted  of  the 
same  three  clinical  transfusion  cases;  the  learning  phase  contained  six  cases. 

One  hundred  and  one  students  were  included  in  the  final  analysis.  Analysis  of 
covariance  was  used  to  test  for  the  presence  of  main  effects  or  an  interaction.  The 
ANCOVA  yielded  a  significant  main  effect  (p=.008)  on  posttest  performance  for  the 
version  of  program  treatment  variable.  No  significant  differences  in  achievement 
were  found  for  method  of  instruction,  nor  was  an  interaction  present  between  the  two 
treatment  variables. 

This  research  suggests  that  advanced  learners  did  not  benefit  equally  from  the 
two  computer  programs.  Learners  using  the  base  computer  program  performed 
significantly  better  than  students  using  the  generative  version  on  the  posttest.  Also, 
although  the  students  felt  there  was  an  advantage  to  working  with  a  peer,  no 
significant  advantage  materialized.  Recommendations  for  future  research  based  on 
these  findings  are  presented. 

This  abstract  accurately  reflects  the  content  of  the  candidate's  thesis.  I  recommend  its 
publication. 
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R.  Scott  Grabingdr 
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CHAPTER  1 


STATEMENT  OF  THE  PROBLEM 

Introduction 

A  common  criticism  of  the  traditional  model  of  teaching  and  learning  involves 
the  lack  of  effective  instructional  methods  to  facilitate  critical  thinking  and  higher 
levels  of  learning  (Bransford,  Franks,  Vye,  &  Sherwood,  1989;  Grabinger,  1996; 
Resnick,  1987;  Scardamalia  &  Bereiter,  1994;  Spiro,  Coulson,  Feltovich,  & 

Anderson,  1988;  Weinstein,  1978).  The  goal  of  instruction  in  the  traditional  model  of 
education  involves  efficiently  and  effectively  transferring  knowledge  to  learners  by 
breaking  instruction  down  into  simple,  basic  units  devoid  of  content  and  context 
(Bednar,  Cunningham,  Duffy,  &  Perry,  1992).  This  form  of  teaching  produces 
students  who  “treat  information  as  facts  to  be  memorized  and  recited  rather  than  as 
tools  to  solve  problems...”  (Grabinger,  1996,  p.  666),  which  results  in  a  use-of- 
knowledge  gap  (Perkins,  1992).  The  traditional  model  of  teaching,  therefore, 
produces  learners  unable  to  recall,  use,  or  transfer  knowledge  and  skills  to  new  and 
novel  situations.  Whitehead  (1929)  calls  this  “inert  knowledge”  since  it  is 
information  students  in  fact  possess,  but  because  it  is  memorized  devoid  of  context,  is 
not  transferable,  even  to  relevant  situations  (Bransford  et  al.,  1989;  Bransford  &  Vye, 
1989;  Cognition  and  Technology  Group  [CTG],  1992;  Grabinger,  1996;  Perkins, 
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1992;  Whitehead,  1929).  Thus,  due  to  the  absence  of  context,  inert  knowledge  maybe 
the  predominant  form  of  knowledge  acquired  by  students  in  the  traditional  model  of 
education. 

In  contrast  to  the  traditional  model  of  education,  many  educators  and  scholars 
believe  that  today’s  complex  world  requires  the  ability  to  use  tools  and  knowledge  in 
a  variety  of  domains  and  situations.  These  critical-thinking  skills  allow  people  to 
fully  participate  in  our  modem,  information-age  society  by  giving  them  the  ability  “to 
analyze  and  synthesize  information  to  solve  technical,  social,  economic,  political,  and 
scientific  problems”  (Grabinger,  1996,  p.  665).  The  question  then  becomes  how  to 
encourage  critical-thinking  skills  development  in  students  rather  than  the  formation  of 
inert  knowledge.  Resnick  (1987)  suggests  that  the  key  to  developing  critical-thinking 
ability  involves  using  instructional  strategies  that  (a)  emphasize  socially  shared 
intellectual  work  organized  around  joint  task  accomplishment,  (b)  encourage  student 
observation  and  commentary  which  makes  usually  covert  cognitive  processes  overt, 
and  (c)  tailor  the  treatment  of  the  subject  matter  to  engage  students  in  the  processes  of 
meaning  construction  and  interpretation.  Such  student-centered  strategies  increase 
time  on  task,  increase  the  amount  and  relevance  of  feedback  to  each  student,  use  a 
performance-based  system  to  assess  student  achievement  (rather  than  a  system  where 
all  students  spend  the  same  amount  of  time  studying  instructional  material),  and 
incorporate  self-paced  instruction  geared  to  student  capabilities  rather  than  class 
average  (Reiser  &  Salisbury,  1995).  This  approach  to  learning  promotes  achievement 
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of  the  basic  goals  of  education  -  retention,  understanding,  and  active  use  of  the 

knowledge  and  skills  (Perkins,  1992).  Perkins  (1992)  writes: 

Surely  we  want  what  is  taught  retained,  else  why  teach  it?  Unless  knowledge 
is  understood,  to  what  purposes  can  it  be  put?  Finally,  having  and 
understanding  knowledge  and  skills  come  to  naught  unless  the  learner  actually 
makes  active  use  of  them  later  in  life.  (p.  18) 

A  student-centered  approach  considers  learning  to  be  an  experiential  process 
(Dewey,  1938)  that  promotes  the  social  structures  and  dynamics  required  for 
knowledge  building  to  occur  among  students  (Scardamalia  &  Bereiter,  1994).  Rather 
than  focusing  on  the  teacher-centered  model  of  education,  which  is  based  on  the 
assumption  that  students  will  learn  because  they  are  asked  or  told  to  learn  (Weinstein, 
1978),  a  student-centered  approach  focuses  on  students  sharing  perspectives  with  the 
teacher  and  other  students  and  then  modifying  internal  representations  of  knowledge 
in  response  to  this  sharing  experience.  In  a  student-centered  environment,  learning 
becomes  a  process  of  constructing,  interpreting,  and  modifying  representations  of 
reality  based  on  experience  (Jonassen,  1994).  Wilson  (1996)  defines  such  a  student- 
centered  learning  environment  as  “a  place  where  learners  may  work  together  and 
support  each  other  as  they  use  a  variety  of  tools  and  information  resources  in  their 
guided  pursuit  of  learning  goals  and  problem-solving  activities”  (p.  5). 
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Purpose  of  the  Study 


Problem  Statement 

For  many  years  now,  research  (Feltovich,  Spiro,  &  Coulson,  1989;  Spiro  et  al., 

1988;  Spiro,  Vispoel,  Schmitz,  Samarapungavan,  &  Boerger,  1987)  has  been 

conducted  on  learning  at  the  stage  of  advanced  knowledge  acquisition  (i.e.,  the  post 

introductory  or  intermediate  stage  of  learning  in  a  subject  domain).  The  goals  of 

advanced  knowledge  acquisition  shift  from  recognition  and  recall  to  attaining  a  deep 

understanding  and  flexible  application  of  content  material  (Spiro  et  al.,  1988); 

however  these  goals  have  been  hard  to  reach.  This  has  been  especially  true  of 

advanced  students  trying  to  leam  in  ill-structured  knowledge  domains  (e.g., 

medicine).  For  example,  Feltovich,  Spiro,  and  Coulson  (1989)  found  that  medical 

students  had  problems  transferring  knowledge  previously  learned  in  one  context,  such 

as  medical  school  coursework,  to  new  situations,  such  as  clinical  problem  solving. 

Spiro  and  his  colleagues  attribute  the  deficiencies  in  learning  outcomes  (i.e.,  inert 

knowledge  formation)  of  students  at  this  advanced  stage  of  learning  to 

oversimplification  (reductive  bias)  of  complexity. 

A  predominant  share  of  the  misconceptions  (and  networks  of  misconception) 
that  we  have  identified  reflect  one  or  another  kind  of  oversimplification  of 
complex  material.. .  .Misconceptions  of  advanced  material  result  from  both 
interference  from  earlier,  simplified  treatments  of  that  material  and  from  a 
prevailing  mode  of  approaching  the  learning  process  in  general  that  fosters 
simplificational  strategies  and  leaves  learners  without  an  appropriate  cognitive 
repertoire  for  the  processing  of  complexity.  (Spiro  et  al.,  1988,  p.  376) 
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Thus,  ill-structuredness  (the  combination  of  breadth,  complexity,  and  irregularity  of  a 
content  domain)  and  the  goals  of  advanced  knowledge  acquisition  unite  to  create  a 
difficult  problem  for  both  teachers  and  students  who  tend  to  rely  on  simple  strategies 
to  present  and  learn  complex  subject  matter. 

Problem  Solution 

In  response  to  the  problem  of  reductive  bias,  Spiro  and  his  colleagues  (1988) 
proposed  cognitive  flexibility  theory  as  a  means  to  create  learning  environments  that 
provide  the  complexity  and  multidimensionality  of  content  required  for  advanced 
knowledge  acquisition  in  ill-structured  domains.  This  study  investigated  the  effects 
of  a  computer-based  microworld  on  knowledge  acquisition  among  advanced  learners. 
The  computer  program  was  developed  according  to  principles  of  learning,  instruction, 
and  knowledge  representation  espoused  by  cognitive  flexibility  theory.  The  program 
was  further  enriched  with  embedded  generative  learning  strategies  (explanation  and 
summary  cues)  and  collaborative  activity  to  produce  a  computer-based  microworld 
that  presents  information,  stimulates  exploration,  facilitates  high  level  thought 
processes,  and  promotes  collaboration  (Grabinger,  1996).  The  remainder  of  this 
chapter  briefly  outlines  cognitive  flexibility  theory,  generative  learning  theory,  and 
collaborative  learning,  frames  the  research  questions,  and  provides  an  overview  of  the 
research  methodology. 
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Cognitive  Flexibility  Theory 

Cognitive  flexibility  theory  (CFT)  targets  the  stage  of  advanced  knowledge 
acquisition,  and,  as  such,  provides  a  framework  for  addressing  the  difficulties 
inherent  in  complex  and  ill-structured  knowledge  domains  (Spiro  et  al.,  1988;  Spiro, 
Feltovich,  Coulson,  &  Anderson,  1989;  Spiro  &  Jehng,  1990).  Evidence  suggests 
that  learners  develop  conceptual  misunderstandings  during  the  advanced  phase  of 
learning,  which  seriously  affects  their  ability  to  transfer  knowledge  to  new  situations 
(Jonassen,  Ambruso,  &  Olesen,  1992;  Spiro  et  al.,  1988).  Cognitive  flexibility  theory 
emphasizes  case-based  instruction  with  multiple  representations  of  content  in  ill- 
structured  knowledge  domains  to  avoid  oversimplifying  instruction  (Spiro  &  Jehng, 
1990).  This  enables  learners  to  investigate  the  multiple  perspectives  inherent  in  a 
knowledge  domain  in  an  exploratory  manner  (Jonassen  et  al.,  1992),  which  promotes 
knowledge  acquisition  and  transfer. 

Generative  Learning  Theory 

Generative  learning  theory  is  based  on  the  notion  of  an  active  learner  (Wittrock, 
1974a;  1974b;  1985;  1990;  1992).  In  this  case,  the  learner  works  to  create  meaning 
by  drawing  relationships  between  existing  information  and  experiences  stored  in 
cognitive  structures  and  information  presented  in  the  learning  environment 
(Grabowski,  1997;  Wittrock,  1985).  Wittrock  (1974b)  demonstrates  the  importance 
of  active  learner  participation  in  the  learning  process  by  writing:  “Although  a  student 
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may  not  understand  sentences  spoken  to  him  by  his  teacher,  it  is  highly  likely  that  a 
student  understands  sentences  that  he  generates  himself”  (p.  182).  Thus, 
understanding  cannot  be  directly  transferred  to  students,  but  must  arise  from  the 
interactions  between  the  learner  and  the  environment  (Wittrock,  1974a;  1974b;  1985). 
The  new  meaning  constructed  from  the  interactions  between  the  learner  and  the 
environment  results  in  learning  with  understanding  (Wittrock,  1974a;  1974b;  1985), 
characterized  by  the  creation  of  meaningful  cognitive  structures,  better  retention  and 
retrieval  of  information,  and  better  explanations  of  the  information  (Di  Vesta,  1989; 
Grabowski,  1997). 


Collaborative  Learning 

Collaborative  learning  actively  involves  learners  in  the  construction  of  their 
knowledge,  but,  in  contrast  to  most  applications  of  cognitive  flexibility  theory  and 
generative  learning  theory  which  focus  primarily  on  the  individual  student's  cognitive 
development,  collaborative  learning  promotes  socially  shared  intellectual 
development.  Collaborative  learning  involves  the  instructional  use  of  small  groups 
where  students  work  together  to  maximize  each  other’s  learning  (Johnson  &  Johnson, 
1992).  To  accomplish  the  goal  of  maximizing  learning,  the  students  are  responsible 
for  learning  the  assigned  material  themselves  and  ensuring  that  all  other  members  of 
the  group  learn  the  material  as  well  (Johnson  &  Johnson,  1992).  Such  peer 
groupwork  allows  the  students  to  actively  engage  in  interactive  higher-level  thinking 
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and  problem-solving  activities  (Damon  &  Phelps,  1989)  leading  to  enhanced 
academic  achievement  and  cognitive  growth,  better  attitudes  and  motivation  toward 
learning,  and  a  host  of  positive  social-emotional  benefits  (Johnson  &  Johnson,  1992; 
Nastasi  &  Clements,  1991;  Slavin,  1996). 

Research  Issues 

What  makes  learning  so  hard  and  what  can  be  done  about  it?  The  answers  are 
teacher-centered  education  and  student-centered  education  respectively.  Learning 
requires  intellectual  effort  on  the  part  of  the  learner  (Osborne  &  Wittrock,  1985),  yet 
with  its  emphasis  on  passive  absorption  of  information,  the  teacher-centered  model  of 
education  devalues  the  effort  required.  In  contrast,  the  student-centered  approach  to 
learning  activates  the  learners’  information-processing  strategies  and  stores  of 
relevant  specific  memories  related  to  the  information  to  be  learned  (Wittrock,  1978). 
This  results  in  “mindful”  (Salomon,  1985)  processing.  Thus,  the  student-centered 
approach  values  “effortful  learning”  and  uses  strategies  designed  to  promote 
generative  learning  (i.e.,  knowledge  construction). 

Cognitive  flexibility  theory  is  a  student-centered  approach  specifically  targeted 
toward  knowledge  construction  and  acquisition  by  advanced  learners  in  ill-structured 
knowledge  domains.  Cognitive  flexibility  theory  advocates  creating  case-based 
computer  learning  environments  that  emphasize  the  conceptual  complexities  and 
interrelatedness  of  subject  matter  content  (Spiro  et  al.,  1988).  The  use  of  authentic, 
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multiple  cases  permits  students  to  see  many  contexts  in  which  concepts  occur  or 
apply,  and  index  meaning  accordingly.  Students,  therefore,  develop  the  ability  to 
adaptively  re-assemble  diverse  elements  of  knowledge  in  response  to  changing 
situational  demands  (i.e.,  the  needs  of  a  given  problem-solving  or  knowledge 
application  situation)  (Spiro  &  Jehng,  1990).  Thus,  cognitive  flexibility  theory-based 
(CFT-based)  computer  learning  environments  stimulate  mindful  processing  which,  in 
turn,  facilitates  the  generation  of  fluid,  flexible,  and  usable  knowledge  structures. 

The  research  to  date  (Hartman  &  Spiro,  1989;  Jacobson,  1990;  Jacobson, 
Maouri,  Mishra,  &  Kolar,  1996;  Spiro  et  al.,  1987)  supports  this  premise;  computer 
programs  based  upon  cognitive  flexibility  theory  principles  do  indeed  promote  higher 
levels  of  knowledge  acquisition  and  transfer  (this  research  is  discussed  more  fully  in 
Chapter  Two).  However,  given  the  multitude  of  student-centered  instructional 
strategies  available  that  also  promote  generative  learning,  might  advanced  learners 
benefit  from  the  simultaneous  integration  of  compatible  learning  strategies  in  a  CFT- 
based  environment?  Might  a  synergy  develop  that  results  in  a  greater  degree  of 
mindfulness  and  consequently,  deeper  processing  of  the  content  material?  If  so, 
would  greater  knowledge  acquisition  result? 
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Research  Questions 


This  study  examined  the  effectiveness  of  a  computer-based  microworld  (see 
definition  on  p.  15)  designed  to  integrate  Resnick’s  (1987)  keys  for  developing 
critical  thinking  and  increasing  learning.  Specifically,  this  study  addressed  the 
question:  Would  knowledge  acquisition  among  advanced  learners  increase  if  a 
cognitive  flexibility  theory-based  computer  program  was  enriched  with  (a)  additional 
generative  learning  strategies  designed  to  facilitate  comprehension  and  understanding 
and  (b)  collaborative  activity  to  attend  to  the  social  nature  of  learning  and  the 
synergistic  development  and  application  of  knowledge  among  groups  of  students?  In 
this  context  two  main  questions  were  explored: 

1 .  Would  the  inclusion  of  embedded  explanation  and  summary  cues 
(generative  learning  strategies)  result  in  differences  in  learning  in  a 
CFT-based  computer  micro  world? 

2.  How  would  the  addition  of  collaborative  learning  affect  learning  in  this 
environment? 
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Research  Methodology 


Experimental  Intervention 

This  study  used  a  2  X  2  factorial  design  consisting  of  two  independent  variables: 

(a)  method  of  instruction  (single  learners  versus  collaborative  learners)  and 

(b)  program  version  (CFT  base  program  versus  CFT  generative  program).  This 
design  resulted  in  four  treatments  consisting  of: 

1 .  Single  student,  base  program  (Single/base).  Subjects  in  this  treatment 

used  the  CFT  base  computer  program  (i.e.,  the  computer  program  without  the 
embedded  generative  learning  strategies). 

2.  Single  student,  generative  program  (Single/generative).  Subjects  in  this 
treatment  used  the  CFT  computer  program  with  the  embedded  generative 
learning  strategies.  These  subjects  were  asked  to  explain  the  choices  they 
made  during  the  program,  as  well  as  to  summarize  the  main  ideas  and 
concepts  for  each  case  presented. 

3.  Paired  students,  base  program  (Pair/base).  Subjects  in  this  treatment 
collaborated  with  a  partner  while  using  the  CFT  base  computer  program. 

4.  Paired  students,  generative  program  (Pair/generative).  Subjects  in  this 
treatment  collaborated  with  a  partner  while  using  the  CFT  generative 
computer  program.  These  students  were  asked  to  explain  and  summarize  in 
the  same  manner  as  the  single/generative  subjects. 
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Dependent  Variable 


The  dependent  variable  for  this  study  was  the  knowledge  constructed 
(i.  e.,  acquired)  by  advanced  learners.  Subjects  took  a  posttest  consisting  of  three 
transfusion  medicine  cases  similar  in  structure  and  content  to  the  six  cases  contained 
in  the  practice  segment  of  the  computer  program.  However,  the  present  study 
departed  from  the  traditional  practice  of  requiring  the  students  assigned  to  the 
collaborative  condition  to  test  individually  based  on  the  following  rationale: 

1 .  Collaborative  learning  is  presumed  to  provide  positive  group-to-individual 
transfer.  “Group-to-individual  transfer  occurs  when  students... demonstrate 
mastery  of  the  material  being  studied  on  a  subsequent  test  taken  individually” 
(Yager,  Johnson,  &  Johnson,  1985,  p.  61).  Research  on  collaborative, 
computer-based  instruction  generally  supports  this  assertion;  students  working 
collaboratively  in  computer-based  instruction  perform  as  well  as  students 
working  alone  (Carrier  &  Sales,  1987;  Makuch,  Robillard,  &  Yoder,  1992; 
Webb,  1985;  1987;  1989)  and  often  perform  better  (Dalton,  Hannafin,  & 
Hooper,  1989;  Hooper,  Temiyakam,  &  Williams,  1993;  Johnson,  Johnson,  & 
Stanne,  1985;  1986). 

2.  Collaborative  learning  is  also  premised  on  the  notion  that  collaborative 
activity  leads  to  emergent  knowledge  -  a  level  of  knowledge  that  results  from 
the  interaction  between  the  understandings  of  the  group  -  rather  than  a 
summation  of  the  knowledge  of  the  individual  participants  (Whipple,  1987). 
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Scaffolding  enables  the  group  to  solve  a  problem  or  achieve  a  goal  which  is 
beyond  the  capabilities  of  the  individuals  involved.  If,  in  fact,  collaboration 
leads  to  higher  performance  of  the  group,  why  test  collaborating  students 
individually?  After  all,  research  demonstrates  that  group-to-individual 
transfer  occurs;  therefore,  if  the  learning  experience  is  positive  for  students 
working  individually,  the  learning  experience  should  also  be  positive  for  the 
students  working  in  groups. 

3.  The  computer  program  used  in  this  study  was  entirely  self-contained. 

The  learning  and  testing  phases  of  the  program  were  developed  and  delivered 
as  one  complete  module,  with  the  pretest  and  posttest  immediately  preceding 
and  following  the  learning  phase  respectively.  Factor  in  the  medical  students’ 
tight  schedules  and  it  would  have  been  awkward  and  inexpedient  to  require 
separate  testing.  Furthermore,  sentiment  exists  among  some  researchers  that 
promoting  shared  cognitive  activity  but  assessing  learning  individually  in  the 
same  study  leads  to  inconsistencies  in  results  (Forman  &  Kraker,  1985).  This 
study,  therefore,  accepted  the  assumptions  of  a  positive  learning  experience 
for  the  students  working  individually  and  that  transfer  would  occur  for  the 
collaborative  pairs.  This  provided  the  opportunity  to  compare  individual 
versus  socially-mediated  knowledge  acquisition. 
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Definition  of  Terms 


•  Advanced  knowledge  -  “learning  beyond  the  introductory  stage  for  a  subject 
area,  but  before  the  achievement  of  practiced  expertise  that  comes  with  massive 
experience”  (Spiro  et  al.,  1988,  p.  375). 

•  Case  -  “a  contextualized  piece  of  knowledge  representing  an  experience  that 
teaches  a  lesson  fundamental  to  achieving  the  goals  of  the  reasoner”  (Kolodner, 
1993,  p.  13). 

•  Case-based  learning  -  involves  problem  solving,  understanding,  and  learning. 
Case-based  learning  “can  mean  adapting  old  solutions  to  meet  new  demands, 
using  old  cases  to  explain  new  situations,  using  old  cases  to  critique  new 
solutions,  or  reasoning  from  precedents  to  interpret  a  new  situation  (much  as 
lawyers  do)  or  create  an  equitable  solution  to  a  new  problem”  (Kolodner, 

1993,  p.  4). 

•  Cognitive  flexibility  -  “the  ability  to  spontaneously  restructure  one’s  knowledge, 
in  many  ways,  in  adaptive  response  to  radically  changing  situational  demands 
(both  within  and  across  knowledge  application  situations)”  (Spiro  &  Jehng, 

1990,  p.  165). 

•  Cognitive  flexibility  theory  -  “a  conceptual  model  for  instruction  that  is  based 
upon  cognitive  learning  theory.  Its  intention  is  to  facilitate  the  advanced 
acquisition  of  knowledge  to  serve  as  the  basis  for  expertise  in  complex  and  ill- 
structured  knowledge  domains”  (Jonassen  et  al.,  1992,  p.  312). 
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•  Collaborative  learning  -  “Collaboration  is  a  coordinated,  synchronous  activity 
that  is  the  result  of  a  continued  attempt  to  construct  and  maintain  a  shared 
conception  of  a  problem”  (Teasley  &  Roschelle,  1993,  p.  235).  Collaboration 
involves  a  coordinated  and  mutual  effort  to  problem  solve  (Teasley  &  Roschelle, 
1993). 

•  Comprehension  -  results  from  formulating  connections,  rather  than  from  placing 
information  or  transforming  information  in  memory.  “The  subtle  difference  lies 
in  the  creation  of  new  understanding  of  the  information  by  the  learner,  rather  than 
changing  the  presented  information”  (Grabowski,  1996,  p.  898). 

•  Computer-based  microworlds  -  computer  programs  that  are  “a  small  but 
complete  subset  of  reality  in  which  one  can  go  to  learn  about  a  specific  domain 
through  personal  discovery  and  exploration”  (Rieber,  1992,  p.  94).  Computer- 
based  microworlds  permit  students  to  explore,  experience  phenomena,  and 
formulate  hypotheses  (Hannafin,  Hannafin,  Hooper,  Rieber,  &  Kini,  1996). 
Computer-based  microworlds  present  information,  stimulate  exploration,  facilitate 
high  level  thought  processes,  and  promote  collaboration  (Grabinger,  1996). 

•  Cooperative  learning  -  cooperative  learning  involves  placing  students  in  small 
groups  to  facilitate  working  toward  common  goals  (Johnson  &  Johnson,  1992; 
Nastasi  &  Clements,  1991).  Cooperative  work  involves  an  activity  where, 
through  division  of  labor,  each  participant  is  responsible  for  a  portion  of  the 
problem  solving  (Teasley  &  Roschelle,  1993). 


15 


•  Generative  learning  activities  -  require  students  to  work  with,  manipulate,  and 
change  information,  relate  information  to  existing  knowledge  structures,  and  use 
information  to  support  problem  solving  (Dunlap  &  Grabinger,  1996). 

•  Generative  learning  theory  -  is  based  on  the  assumption  that  the  learner  is  an 
active  participant  in  the  learning  process  where  “comprehension  and 
understanding  result  from  the  processes  of  generating  relations  both  among  and 
between  experience  or  prior  learning  and  new  information”  (Wittrock,  1992, 

p.  532). 

•  Ill-structured  knowledge  domains  -  knowledge  domains  in  which  “many 
concepts  (interacting  contextually)  are  pertinent  in  the  typical  case  of  knowledge 
acquisition,  and  their  patterns  of  combination  are  inconsistent  across  case 
applications  of  the  same  nominal  type”  (Spiro  et  al.,  1988  ,  p.  375).  Examples 
include  history,  biomedicine,  and  literary  interpretation  (Jacobson  &  Spiro,  1994). 

•  Inert  knowledge  -  “is  knowledge  that  can  usually  be  recalled  when  people  are 
explicitly  asked  to  do  so  but  that  is  not  used  spontaneously  in  problem-solving 
contexts  even  though  it  is  relevant”  (CTG,  1992,  p.  136).  Inert  knowledge  is 
passively  learned  by  students  and  available  for  tests,  but  not  available  for 
application  in  new  situations  (Bransford  et  al.,  1989). 

•  Learning  with  understanding  -  Wittrock  (1974a)  writes:  “Learning  with 
understanding,  which  is  defined  by  long-term  memory  plus  transfer  to 
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conceptually  related  problems,  is  a  process  of  generating  semantic  and  distinctive 
idiosyncratic  associations  between  stimuli  and  stored  information”  (p.  89). 

•  Mindfulness  -  “the  voluntary,  controlled  employment  of  nonautomatic 
processing  operations. ...It  is  manifested  in  attention  to  details,  in  the  careful 
examination  of  a  problem’s  given  conditions,  in  consideration  of  alternatives,  in 
the  generation  of  hypotheses  and  inferences,  in  the  reading  of  text  for  deeper 
meanings,  in  linking  new  information  to  remote  knowledge  structures,  in 
processes  of  abstraction  and  decontextualization,  and  the  like”  (Salomon,  1985, 

P-  213). 

•  Reductive  bias  -  the  tendency  to  oversimplify  important  aspects  of  complexity 
resulting  in  misconceptions  of  advanced  material.  Reductive  bias  “leaves  learners 
without  an  appropriate  cognitive  repertoire  for  the  processing  of  complexity” 
(Spiro  et  al.,  1988).  Spiro  and  his  colleagues  identify  the  following  forms  of  bias: 
(a)  oversimplification  of  complex  and  irregular  structure,  (b)  overreliance  on  a 
single  basis  for  mental  representation,  (c)  overreliance  on  top  down  processing, 

(d)  overreliance  on  precompiled  knowledge  structures,  (e)  context-independent 
conceptual  representation,  (f)  rigid  compartmentalization  of  knowledge  structures, 
and  (g)  passive  transmission  of  knowledge. 
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CHAPTER  2 


REVIEW  OF  LITERATURE 

Introduction 

Since  the  1930s,  competitive  and  individualistic  learning  has  dominated  all 
levels  of  education  (Johnson  &  Johnson,  1992).  This  view  of  education  (commonly 
called  the  traditional  model  of  education)  assumes  learning  is  a  passive,  individual 
activity  (Brown  &  Campione,  1996),  where  the  teacher  simply  transmits  knowledge 
to  students  as  efficiently  as  possible  (Bednar  et  al.,  1992).  To  accomplish  this, 
educators  simplify  instruction  by  neatly  defining  and  prescribing  subject  matter 
domains  in  order  to  teach  only  the  critical  attributes  of  the  domains  (Duffy  &  Rnuth, 
1990).  The  traditional  model  of  education,  therefore,  assumes  (a)  that  learning 
involves  forming  simple  associations  based  upon  external  reinforcement  (Brown  & 
Campione,  1996)  and  (b)  that  learning  occurs  most  efficiently  if  the  excess  baggage 
of  irrelevant  content  and  context  are  eliminated  (Bednar,  Cunningham,  Duffy,  & 
Perry,  1995). 

Many  researchers  believe  the  traditional  approach  to  education  is  obsolete 
(Bednar  et  al.,  1992;  1995;  Bransford  et  al.,  1989;  Bransford  &  Vye,  1989;  CTG, 
1992;  Grabinger,  1996;  Perkins,  1992;  Reigeluth,  1995;  Resnick,  1987;  1989; 
Scardamalia  &  Bereiter,  1994).  A  major  criticism  focuses  on  the  assumption  that 
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learning  occurs  devoid  of  context.  Whitehead  (1929)  believed  the  lack  of  relevant 
context  provided  by  the  traditional  model  of  education  produces  inert  knowledge. 
Inert  knowledge  is  knowledge  that  students  possess,  but  not  used  in  problem-solving 
contexts,  even  when  relevant,  unless  the  students  are  specifically  asked  to  do  so 
(Bransford  et  al.,  1989;  Bransford  &  Vye,  1989;  CTG,  1992).  Inert  knowledge  exists 
as  an  “island  of  information,”  which  while  retrievable,  provides  little  value  to  the 
learner  for  interpreting,  modifying,  or  influencing  performance  (Hannafin,  1992). 
John  Dewey  similarly  criticized  the  traditional  approach  to  education  because 
teachers  present  dry  problems  of  little  relevance  to  students  (Phillips  &  Soltis,  1991). 
Dewey  believed  information  presented  this  way  is  committed  to  memory  as  “static, 
cold-storage  knowledge”  and  unless  students  have  the  opportunity  to  use  this 
information  in  meaningful  problem-solving  activities,  the  information  remains 
“sterile”  (Phillips  &  Soltis,  1991).  Removing  the  context,  therefore,  limits  students’ 
abilities  to  learn  and  apply  concepts  (i.  e.,  successful  learning). 

Successful  learning  requires  more  than  literal  encoding  of  formal  instruction  in  a 
context-free  environment  (Hannafin,  1992).  Successful  learning  occurs  by  engaging 
in  activities  that  encompass  the  concepts  students  are  trying  to  learn  (Bransford  et  al., 
1989;  Brown,  Collins,  &  Duguid,  1989;  CTG,  1992;  Spiro  et  ah,  1988).  Successful 
learning  occurs  when  students  retain,  understand,  and  actively  use  the  knowledge  and 
skills  learned  -  Perkins’  goals  of  education  -  in  the  world  beyond  the  school  walls. 
Perkins  (1992)  defines  these  goals  as  generative  knowledge;  “knowledge  that  does 
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not  just  sit  there  but  functions  richly  in  people’s  lives  to  help  them  understand  and 
deal  with  the  world”  (p.  5).  Generative  knowledge  requires  existing  knowledge  to  be 
evaluated  concurrently  with  new  knowledge  and  reconstructed  accordingly,  that 
knowledge  be  assimilated,  and  that  perceptions  of  meaning,  value,  and  importance  be 
derived  (Hannafm,  1992;  Hannafm  &  Rieber,  1989).  Generative  knowledge 
therefore,  is  the  antithesis  of  inert  knowledge.  Thus,  how  can  educators  promote  the 
development  of  generative  knowledge?  What  theories  or  instructional  methods  exist 
to  promote  the  development  of  generative  knowledge? 

Current  cognitive  theory  (Bednar  et  al.,  1992;  1995;  Bransford  et  al.,  1989; 
Bransford  &  Vye,  1989;  Brown  et  al.,  1989;  CTG,  1992;  Duffy  &  Knuth,  1990; 
Grabinger,  1996;  Perkins,  1992;  Jonassen,  1991;  Resnick  1987;  1989;  Spiro  et  al., 
1988)  emphasizes  three  essential  and  interrelated  aspects  of  learning:  (a)  learning 
involves  knowledge  construction  rather  than  knowledge  absorption  and  recording, 

(b)  learners  build  on  current  knowledge  to  construct  new  knowledge  (i.e.,  learning  is 
knowledge  dependent),  and  (c)  learning  depends  on  the  context  or  situation  in  which 
it  occurs.  Many  researchers  (Bransford  et  al.,  1989;  Brown  &  Campione,  1996; 
Grabinger,  1996;  Hannafm,  1992;  Jonassen,  1991;  Spiro  et  al.,  1988;  Wilson,  1996) 
advocate  creating  student-centered  learning  environments  that  embody  the  three 
principles  of  learning  just  discussed.  This  chapter  presents  a  selective  review  of  the 
research  and  theory  of  three  strategies  designed  to  capture  the  complexities  of 
learning  and  promote  generative  learning. 
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The  chapter  begins  by  examining  cognitive  flexibility  theory  (CFT),  a 
conceptual  model  for  instruction  that  facilitates  advanced  knowledge  acquisition 
(Spiro  et  al.,  1988)  by  activating  generative  learning  processes.  The  chapter  then 
moves  to  a  discussion  of  generative  learning  theory,  which  assumes  that  learners 
actively  participate  in  the  learning  process  by  constructing  meaningful 
understanding  of  information  found  in  the  learning  environment  (Wittrock,  1974a; 
1974b).  Next,  research  on  collaborative  learning  is  reviewed.  The  reviews  of 
literature  provide  the  rationale  for  the  development  of  the  student-centered  computer 
programs  used  in  this  study.  The  chapter  concludes  by  restating  the  research 
questions  and  offering  the  hypotheses  of  interest. 

Cognitive  Flexibility  Theory:  Assumptions  and  Theory 
Spiro  et  al.  (1988)  believe  that  schools  simplify  representations  of  knowledge  to 
allow  students  to  see  the  main  point.  Simplifying  the  content  removes  the  context, 
which  rather  than  simplifying  the  concept,  results  in  teaching  a  different  concept. 
Spiro  and  his  colleagues  (1988)  argue  that  case-based  learning  captures  the 
complexity  of  real  experiences.  Students  work  through  several  cases  examining  the 
concept  from  different  points  of  view  in  different  contexts.  This  allows  the  students 
to  see  multiple  contexts  in  which  the  concept  occurs  and  applies,  and  thus  index  its 
meaning. 
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Cognitive  flexibility  theory  is  based  on  two  elements:  (a)  that  learning  occurs 
along  a  continuum  from  novice  to  expert  and  (b)  that  knowledge  domains  can  be 
defined  by  two  conceptual  characteristics  -  increased  complexity  and  increased  ill¬ 
structuredness  (Jacobson  &  Spiro,  1994;  Spiro  et  al.,  1988;  Spiro,  &  Jehng,  1990). 
Cognitive  flexibility  theory  posits  that  knowledge  acquisition  follows  a  continuum 
from  beginner  to  expert,  but  focuses  on  the  intermediate  or  advanced  learning  stage. 
Introductory  learning  typically  involves  learners  with  very  little  transferable 
knowledge  in  a  content  area  (Jonassen  &  Grabinger,  1993).  The  goal  of  instruction 
for  beginning  learners  involves  providing  a  general  orientation  and  exposure  to 
content  followed  by  objective  assessment  measuring  recognition  and  recall  of  facts 
(Spiro  et  al.,  1988).  In  contrast,  during  the  intermediate  stage  of  learning,  learners 
acquire  advanced  knowledge  in  order  to  solve  complex  domain-oriented  problems 
(Jonassen  &  Grabinger,  1993).  During  the  advanced  learning  phase  the  learning 
goals  shift  to  (a)  mastery  of  content  complexity  (i.e.,  acquisition  of  the  conceptual 
complexity  necessary  for  understanding  important  concepts)  and  (b)  knowledge 
applicability  (i.e.,  the  ability  to  adaptively  transfer  knowledge  to  novel  and  realistic 
situations)  (Spiro  et  al.,  1989).  Advanced  knowledge  acquisition,  therefore,  involves 
attaining  a  deeper,  richer  understanding  of  content  material,  as  well  as  the  ability  to 
intelligently  reason  with  and  apply  it  in  diverse  contexts  (Jacobson  &  Spiro,  1994; 
Spiro  et  al.,  1988). 
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The  second  distinguishing  element  of  cognitive  flexibility  theory  involves  the 
conceptual  characteristics  of  increased  domain  complexity  and  increased  ill¬ 
structuredness  (Jacobson  &  Spiro,  1994;  Jonassen  et  al.,  1992;  Spiro  et  al.,  1987; 
1988;  1989;  Spiro  &  Jehng,  1990).  Complexity  occurs  when  the  domain  content 
places  unusual  demands  on  cognition  as  compared  to  the  cognitive  loading  in  the 
introductory  phase  of  learning  (Feltovich  et  al.,  1989).  Complexity  can  result  from 
(a)  task  multidimensionality,  which  places  unusual  or  excessive  demands  on  working 
memory,  (b)  the  abstractness  of  the  content  or  the  semantic  difference  between 
concepts  and  their  formal  symbolic  representation,  (c)  heavy  reliance  on  the  learner's 
prior  knowledge  (which  may  be  faulty  or  at  odds  with  the  concepts),  or  (d)  irregular, 
inconsistent  concepts  or  the  interaction  of  many  concepts  (Feltovich  et  al.,  1989; 
Jonassen  et  al.,  1992). 

Related  to  domain  complexity  is  the  notion  of  ill-structuredness  (Jacobson  & 
Spiro,  1994;  Jonassen  et  al.,  1992;  Spiro  et  al.,  1987;  1988;  1989;  Spiro  &  Jehng, 
1990).  “By  ill-structuredness  we  mean  that  many  concepts  (interacting  contextually) 
are  pertinent  in  the  typical  case  of  knowledge  acquisition,  and  that  their  patterns  of 
combination  are  inconsistent  across  case  applications  of  the  same  nominal  type” 
(Spiro  et  al.,  1988,  p.  375).  Characteristics  of  an  ill-structured  knowledge  domain 
include  the  following: 

•  no  general  rules  or  defining  characteristics  exist  that  cover  most  cases, 

•  inconsistent  hierarchies  of  relationships  exist  among  cases, 
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•  the  same  features  of  cases  assume  different  levels  of  importance  in 
different  contexts, 

•  no  prototypic  cases  exist  or  they  are  misleading,  and 

•  interactions  among  features  cause  case  novelty. 

(Spiro  et  al.,  1987;  Jonassen  et  al.,  1992). 

Ill-structured  knowledge  domains  place  demands  on  learners  that  are  at  odds  with  the 
cognitive  modes  and  instructional  practices  appropriate  for  introductory  learning 
(Feltovich  et  al.,  1989). 

The  notion  that  advanced  learning  differs  from  introductory  learning  with 

respect  to  goals,  complexity,  and  ill-structuredness  suggests  different  approaches  to 

learning  are  necessary.  However,  all  too  often  the  same  teaching  methods  and  tactics 

used  for  initial  knowledge  acquisition  are  also  used  for  intermediate  learning. 

Learning  and  instruction  for  mastery  of  complexity  and  application  in  a 
complex  and  ill-structured  domain  cannot  be  compartmentalized,  linear, 
uniperspectival,  neatly  hierarchical,  simply  analogical,  or  rigidly  prepacked. 
Yet  it  much  too  often  is,  and  the  result  is  the  development  of  widespread  and 
serious  misconceptions  and  difficulties  in  knowledge  application.  (Spiro  & 
Jehng,  1990,  p.  168) 

Teaching  methods  geared  toward  introductory  learning  lead  to  oversimplification 
(reductive  bias)  of  complex  material  (Spiro  et  al.,  1988)  which  contributes 
significantly  to  examples  of  learning  failure  (Spiro  et  al.,  1988;  Jacobson  &  Spiro, 
1995).  Cognitive  flexibility  theory  offers  several  principles  designed  to  overcome 
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reductive  bias  and  promote  the  goals  of  advanced  knowledge  acquisition  (Spiro  et  al., 
1988): 

1 .  Avoid  oversimplification  and  overregulation.  Advanced  knowledge 
acquisition  requires  that  learners  be  made  aware  that  knowledge  is  not  simple 
and  orderly,  but  complex  and  irregular.  “It  is  important  to  lay  bare  the 
limitations  of  initial,  first  pass  understandings,  to  highlight  exceptions,  to 
show  how  the  superficially  similar  is  dissimilar  and  superficial  unities  are 
broken”  (Spiro  et  al.,  1988,  p.  377).  Additionally,  rather  than  decomposing 
and  reassembling  information,  instruction  should  reflect  the  intricate  patterns 
of  component  interactions. 

2.  Provide  multiple  representations  of  content.  Single  depictions  of  complex 
and  ill-structured  knowledge  will  miss  or  misrepresent  important  aspects  of 
complex  concepts.  Cognitive  flexibility  requires  the  use  of  multiple  themes, 
concepts,  analogies,  and  points  of  view  (Jacobson  &  Spiro,  1995)  to  provide  a 
diversified  repertoire  of  ways  of  thinking  about  a  conceptual  topic 

(Spiro  et  al.,  1988). 

3.  Use  and  emphasize  cases.  In  complex,  ill-structured  knowledge  domains, 
great  variability  may  exist  between  cases  concerning  the  applicability  of 
relevant  concepts.  Cases  may  not  be  linked  together  by  general  principles, 
therefore,  multiple  cases  are  necessary  to  illustrate  abstract  concepts 
associated  with  ill-structured  knowledge  domains.  “The  more  variegated 
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these  cases  are,  the  broader  the  conceptual  basis  that  they  are  likely  to 
support”  (Jonassen  et  al.,  1992,  p.  312). 

4.  Emphasize  conceptual  knowledge  as  knowledge  in  use.  In  an  ill-structured 
knowledge  domain  the  way  a  concept  is  used  or  applied  may  vary  greatly 
across  cases,  which  makes  it  more  difficult  to  extrapolate  the  concepts  from 
the  features  of  the  cases.  Therefore,  “if  a  concept's  meaning  cannot  be 
determined  universally  across  cases,  then  one  must  pay  attention  to  the  details 
of  how  the  concept  is  used  -  knowledge  in  practice,  rather  than  in  the 
abstract”  (Spiro  et  al.,  1988,  p.  380). 

5.  Emphasize  knowledge  construction  rather  than  knowledge  transmission. 
Given  the  emphasis  placed  on  case-based  learning  and  knowledge  in  use, 
learners  must  be  able  to  assemble  (construct)  meaningful  knowledge 
representations  in  order  to  adaptively  fit  the  situation  at  hand.  In  other  words, 
the  complexity  and  irregularity  inherent  in  ill-structured  domains  requires 
flexible,  recombinable  knowledge  structures.  Thus  the  “storage  of  fixed 
knowledge  is  devalued  in  favor  of  the  mobilization  of  potential  knowledge” 
(Spiro  et  al.,  1987,  p.  181).  Knowledge  construction  implies  active  learner 
involvement  in  knowledge  acquisition  accompanied  by  expert  guidance  and 
cognitive  support. 

6.  Emphasize  noncompartmentalization  of  cases  and  concepts.  The 
complexity  and  irregularity  inherent  in  the  cases  and  examples  in  ill-structured 
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knowledge  domains  precludes  compartmentalizing  knowledge.  “Rather  than 
mapping  knowledge  onto  the  learner,  the  learner  must  map  contexts  onto 
his/her  own  knowledge  as  it  is  being  acquired  in  order  to  support  the 
transferability  of  knowledge”  (Jonassen  et  al.,  1992,  p.  313).  Multiple 
interconnectedness  and  differing  thematic  perspectives  among  different  cases 
and  concepts  enable  the  situation-dependent  and  adaptive  schema  assembly 
that  promote  knowledge  transfer  (Spiro  et  al.,  1988). 

Cognitive  flexibility  theory  prescribes  using  a  case-based  instructional  approach 
that  provides  access  to  thematic  information  structures  to  create  learning 
environments  that  stimulate  the  development  and  application  of  flexible  knowledge 
structures.  Such  environments  stimulate  the  mindful  processing  of  information 
(Salomon,  1985)  necessary  for  generative  learning  to  occur.  In  sum,  by  focusing  on 
students  generating  flexible,  usable  knowledge  in  an  information-rich  learning 
environment,  cognitive  flexibility  theory  is  a  generative  activity. 

Research  on  Cognitive  Flexibility  Theory 

Evidence  validating  the  effectiveness  of  CFT  is  generally  positive  in  the  area  of 
higher  level  thinking  skills.  For  example,  Spiro  et  al.  (1987)  conducted  two  studies 
with  high  school  students  in  which  control  groups  studied  the  same  cases  as  the 
experimental  groups,  but  the  cases  were  presented  linearly,  while  the  cases  for  the 
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experimental  groups  contained  case-to-case  linkages.  Control  group  learners 
outscored  experimental  group  learners  on  reproductive  tests  of  memory, 
but  the  experimental  groups  outscored  the  control  groups  on  six  different  tests  of 
application  and  transfer.  Spiro  and  his  colleagues  (1987)  concluded  that  conventional 
methods  produce  better  results  on  traditional  tests  stressing  rote  memorization,  but  if 
the  goal  of  education  is  generative  knowledge,  “then  it  would  seem  that  methods  like 
ours  are  far  preferable  to  the  conventional  ones”  (p.  191).  Hartman  and  Spiro  (1989) 
expanded  on  this  research  effort  by  studying  the  effects  of  multiple  perspectives, 
flexible  representation,  and  assessment  of  transfer.  They  found  no  differences  in 
reproductive  memory,  but  in  knowledge  application  and  transfer,  the  flexibility  group 
significantly  outperformed  the  control  group.  Finally,  Jacobson  (1990)  compared  a 
linear,  computer-based  drill  treatment  with  an  experimental  treatment  emphasizing 
multiple  representations,  linking  abstract  ideas  to  case  examples,  and  the 
interrelationships  between  surface  and  structural  knowledge.  Again  the  drill  group 
recalled  more  facts,  but  the  experimental  group  attained  higher  scores  on  all  transfer 
tests  of  knowledge. 

More  recent  investigations  concentrate  on  expanding  our  understanding  of 
cognitive  flexibility  theory  and  computer-based  microworlds.  For  example,  Jacobson 
&  Spiro  (1995)  investigated  the  effects  of  learning  from  a  minimal  hypertext/drill 
condition  versus  a  cognitive  flexibility  hypertext.  The  study  consisted  of  two  main 
parts.  The  first  part  was  a  reading  stage  where  both  groups  read  the  same 
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instructional  content.  The  second  stage  consisted  of  a  study  stage,  in  which  the 
control  group  completed  a  computer-based  drill  program  based  on  the  facts  and 
concepts  of  the  reading  stage,  while  the  experimental  group  received  a  hypertext 
treatment  stressing  knowledge  interrelationships  and  knowledge  assembly.  The 
results  revealed  that  the  control  group  performed  higher  on  measures  of  memory  for 
factual  knowledge,  while  the  experimental  group  demonstrated  greater  knowledge 
transfer. 

These  research  efforts  appear  to  validate  the  efficacy  of  cognitive  flexibility 
theory;  instruction  based  upon  CFT  principles  promotes  development  of  certain  types 
of  generative  knowledge.  The  studies  demonstrate  that  CFT-based  learning 
environments  require  students  to  manipulate  and  relate  information  to  existing 
knowledge  structures  which  supports  the  problem-solving  and  application  processes 
endemic  to  advanced  knowledge  acquisition.  However,  opportunities  for  deep 
learning  are  not  always  fully  taken  advantage  of,  even  when  presented  as  unique 
learning  environments  (Salomon,  1985),  such  as  a  cognitive  flexibility  learning 
environment.  In  other  words,  although  a  learning  environment  may  be  based  upon 
cognitive  flexibility  theory  (or  some  other  student-centered  strategy)  students  may  not 
employ  the  mindful  processes  required  of  generative  learning.  Given  the  necessity  to 
activate  mindful  processes  for  generative  learning  to  occur,  the  question  becomes: 
Would  the  integration  of  compatible  generative  learning  activities  into  a  CFT-based 
learning  environment  promote  deeper  processing  of  material,  and  hence,  increased 
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knowledge  acquisition  by  advanced  learners?  The  next  section  of  this  chapter 
discusses  generative  learning  theory  and  collaborative  learning,  two  generative 
activities  that,  in  combination  with  cognitive  flexibility  theory,  should  increase 
learning  by  advanced  learners. 

Generative  Learning  Theory 

Generative  learning  theory  provides  an  understanding  of  the  process  of 
comprehension  (Wittrock,  1992).  Comprehension  requires  learners  to  formulate 
connections  between  the  different  parts  of  the  information  being  perceived  and 
between  that  information  and  what  exists  in  memory  (Wittrock,  1985).  This  causes 
learners  to  reorganize,  elaborate,  and/or  reconceptualize  information  -  rather  than 
stuff  more  information  into  memory  -  which  results  in  meaningful  learning  and 
comprehension  (Grabowski,  1996).  The  important  pedagogical  point  is  that 
knowledge  cannot  be  transferred  directly  from  the  teacher  to  students,  but  has  to  be 
created  within  each  student  (Harlen  &  Osborne,  1985).  Learning  with  understanding, 
therefore,  requires  intellectual  effort  on  the  part  of  the  learner  to  generate 
relationships  between  stimuli  and  stored  information  (Osborne  &  Wittrock,  1985). 
Wittrock  (1992)  sums  up  generative  learning  theory  in  the  following  manner: 
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At  the  essence  of  this  functional  model  are  generative  learning  processes 
people  use  actively  and  dynamically  to  (a)  selectively  attend  to  events  and 
(b)  generate  meaning  for  events  by  constructing  relations  between  new  or 
incoming  information  and  previously  acquired  information,  conceptions,  and 
background  knowledge.  These  active  and  dynamic  generations  lead  to 
reorganizations  and  reconceptualiztions  and  to  elaborations  and  relations  that 
increase  understanding,  (p.  532) 

Wittrock  (1990)  believes  that  there  are  two  categories  of  activities  that  stimulate 

generative  processes.  The  first  category,  designed  to  stimulate  processing  between 

the  different  parts  of  the  information  being  received,  includes  activities  such  as 

composing  titles  and  headings,  writing  questions,  stating  objectives,  writing 

summaries,  drawing  graphs  and  tables,  and  constructing  main  ideas.  The  second 

category  consists  of  demonstrations,  metaphors,  analogies,  examples,  problem 

solving,  explanations,  paraphrases,  and  inferences  which  generate  relationships 

between  the  external  stimulus  (instruction)  and  prior  knowledge.  These  categories 

can  be  used  in  either  a  teacher-provided  or  learner-generated  format  (Grabowski, 

1996;  Wittrock,  1990).  For  example,  the  teacher  can  provide  summaries  or  ask  the 

learners  to  summarize.  However,  Wittrock  (1974a)  cautions  that  understanding 

cannot  be  given  directly  to  students;  if  understanding  is  to  occur,  students  must  make 

the  connections  themselves.  Di  Vesta  (1989)  concurs: 

What  is  learned  is  not  necessarily  that  which  is  stated  in  the  title  of  the  course, 
what  is  described  in  the  curriculum  or  syllabus,  or  the  behavior  described  in  a 
behavioral  objective  or  the  content  of  the  course  defined  by  texts, 
assignments,  delivery  system,  and  curricular  materials.  What  is  learned 
depends  on  processing  requirements  that  are  actually  carried  out  by  the 
learner,  (p.  55) 
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In  summary,  generative  learning  theory  posits  that  learners  put  forth  intellectual 
effort  to  actively  construct  knowledge  by  generating  relationships  (a)  between  the 
different  parts  of  the  incoming  information  and  (b)  between  the  new  information  and 
prior  knowledge  and  other  memory  components  (Grabowski,  1996;  Wittrock,  1974a; 
1974b;  1985;  1990;  1992).  Unfortunately,  most  of  the  research  on  generative 
learning  theory  concentrates  on  reading  comprehension  and  individual  students. 
Fortunately,  the  model  emphasizes  transforming  static  information  (text)  into  flexible, 
usable  knowledge  (Grabinger,  1996),  thus  there  may  be  transferability  to  other 
student-centered  strategies  designed  to  increase  learning. 

Compatibility  of  Cognitive  Flexibility  Theory  and  Generative  Learning  Theory 
Cognitive  flexibility  theory  advocates  constructing/applying  knowledge  in  a 
multiple  case  format  to  encourage  development  of  flexible  representations  of  content 
(Spiro  et  al.,  1988).  Learners  manipulate  the  content  information  to  make  the 
multiple  connections  between  and  among  the  information  and  prior  learning.  Thus 
cognitive  flexibility  theory  is  a  generative  activity  and  conceptually,  very  similar  to 
generative  learning  theory.  However,  the  two  models  differ  with  respect  to  practice. 
First,  there  are  obvious  contextual  differences  between  the  two  theories.  Cognitive 
flexibility  theory  advocates  using  multiple  cases  in  a  complex  learning  environment 
to  increase  learning,  whereas,  generative  learning  theory  prescribes  specific  strategies 
designed  to  increase  student  reading  comprehension  and  understanding  of  text. 
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Second,  Spiro  and  his  colleagues  specifically  target  the  advanced  stage  of  learning  for 
CFT  application,  while  generative  learning  theory  application  appears  unrestricted 
regarding  the  stages  of  learning.  Given  the  conceptual  similarities,  but  practical 
differences,  how  might  cognitive  flexibility  theory  and  generative  learning  theory 
interact  to  affect  learning? 

The  limited  cognitive  flexibility  theory  research  discussed  earlier  demonstrates 
the  effectiveness  of  cognitive  flexibility  theory  in  promoting  advanced  knowledge 
acquisition.  Since  generative  learning  theory  offers  strategies  designed  to  make  the 
learning  processes  explicit  (i.e.,  stimulates  mindful  processes)  and  increase 
comprehension  and  understanding,  it  is  reasonable  to  expect  that  generative  learning 
strategies  would  enrich  a  cognitive  flexibility  learning  environment.  In  other  words, 
using  strategies  to  stimulate  one  or  both  of  the  generative  processes  should  help 
advanced  learners  develop  the  rich  knowledge  structures  cognitive  flexibility  strives 
to  promote.  The  strategies  of  interest  in  this  study  are  summarization,  which 
facilitates  making  organizational  connections  between  the  different  parts  of  the  new 
information,  and  explanation,  which  supports  making  connections  between  prior 
knowledge  and  the  new  information. 


Individual  Student  Focus 


One  interesting  observation  concerning  the  theories  of  learning  just 
reviewed  -  especially  cognitive  flexibility  theory  -  is  the  focus  on  the  individual 
student.  Given  the  proclivity  to  use  computers  to  deliver  CFT-based  instruction,  the 
focus  on  students  interacting  individually  with  computers  to  learn  the  respective 
subject  domains  is  especially  intriguing  since  much,  if  not  most,  research  on 
computer-based  learning  focuses  on  collaborative  learning.  The  concentration  on  the 
individual  learner  within  the  cognitive  flexibility  theory  literature  may  reflect  the 
initial  empirical  evaluation  of  cognitive  flexibility  theory,  or  the  assumption  that 
advanced  learners  (the  stage  of  learning  hypothesized  to  benefit  from  CFT-based 
instruction)  leam  best  individually,  or  perhaps  it  is  based  on  the  belief  that  computers 
represent  the  ideal  medium  to  individualize  instruction.  Whatever  the  reason,  the 
reality  is  that  students  often  work  in  small  groups  at  computers  (Hooper,  1992). 
Therefore,  an  investigation  into  the  effects  of  collaborative  learning  on  cognitive 
flexibility  theory  is  warranted,  especially  since  social  interaction  is  such  an  integral 
part  of  a  student-centered  learning  environment  (Resnick,  1987;  Wilson,  1996). 

Collaborative  Learning 

At  the  outset  it  is  important  to  make  the  distinction  between  the  terms 
“cooperative  learning”  and  “collaborative  learning”  because  they  differ  with  respect 
to  how  they  encourage  student  interaction.  Cooperative  learning  refers  to  group 
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learning  approaches  where  a  division  of  labor  among  participants  requires  individual 
students  to  take  responsibility  for  different  portions  of  the  task  (Crook,  1994;  Damon 
&  Phelps,  1989;  Stodolsky,  1984;  Teasley  &  Roschelle,  1993).  Collaborative 
learning,  on  the  other  hand,  involves  students  working  jointly  in  a  coordinated  effort 
on  the  same  problem  (Damon  &  Phelps,  1989;  Teasley  &  Roschelle,  1993).  In  both 
cases,  the  grouped  students  share  a  common  goal,  interact,  and  contribute  to  the  group 
activity  (Stodolsky,  1984),  but  the  differences  in  task  structure  result  in  differences  in 
equality  and  mutuality  (two  indexes  of  peer  engagement).  “Equality  means  that  both 
parties... take  direction  from  one  another  rather  than  one  party  submitting  to  a 
unilateral  flow  of  direction  from  the  other;  and  mutuality  means  that  the  discourse  in 
the  engagement  is  extensive,  intimate,  and  connected”  (Damon  &  Phelps,  1989, 
p.  10).  According  to  Damon  and  Phelps  (1989),  cooperative  learning  is  high  on 
equality,  but  variable  on  mutuality  due  to  the  task  subdivision  that  requires  learners  to 
accomplish  much  of  their  work  individually.  In  contrast,  students  work  jointly  on  the 
same  problem  in  a  collaborative  environment  creating  “an  engagement  rich  in  mutual 
discovery,  reciprocal  feedback,  and  frequent  sharing  of  ideas”  (Damon  &  Phelps, 
1989,  p.  13).  The  higher  degrees  of  mutuality  and  equality  present  in  a  collaborative 
environment  results  in  interaction  between  (not  summation  of)  the  understandings  of 
the  students  which  leads  to  emergent  knowledge  (Whipple,  1987). 

Collaborative  learning  is  rooted  in  the  assumption  that  knowledge  is  socially 
constructed  and  that  social  interaction  is  necessary  for  learning  to  occur  (Vygotsky, 
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1978).  Two  different  theoretical  perspectives  offer  possible  explanations  for  the 
cognitive  growth  associated  with  collaborative  learning.  The  first,  socio-cognitive 
theory,  posits  that  interindividual  conflict  facilitates  cognitive  growth  (Nastasi  & 
Clements,  1992;  1993).  Development  occurs  as  students  enter  into  a  conversational 
process  of  negotiation  and  justification  in  an  attempt  to  resolve  differences  of 
agreement  (Crook,  1994;  Nastasi  &  Clements,  1992;  1993).  As  the  students 
reevaluate  their  positions  and  seek  new  information  to  resolve  conflict,  they  learn  the 
material  better  (Webb,  1987)  and  restructure  old  knowledge  structures  into  new 
knowledge  structures  (Hanafm,  1992).  Co-construction  is  the  second  theoretical 
explanation  for  improved  achievement.  In  this  case  interpersonal  conflict  is  absent 
and  partners  integrate  their  differing  task  conceptualizations  into  a  mutual  plan  for 
solving  a  problem  neither  could  solve  alone  (Nastasi  &  Clements,  1993).  Thus 
collaborating  students  scaffold  or  guide  and  correct  each  other  and  build  on  each 
other's  ideas  until  they  reach  the  solution.  In  either  case,  the  process  of  collaborating 
requires  students  to  make  their  thinking  public  and  explicit  (Crook,  1994),  which 
facilitates  intellectual  growth  through  a  process  of  declaring  and  justifying  ideas, 
opinions,  and  interpretations  (Damon  &  Phelps,  1989).  Collaborating  students, 
therefore,  “retrieve  prior  knowledge,  seek  new  information,  evaluate  their  own  and 
others'  answers,  ideas,  and  opinions,  confront  their  own  misunderstandings  and  lack 
of  knowledge,  and  as  a  consequence,  restructure  their  thinking”  (Webb  &  Lewis, 
1988,  p.  181)  which  leads  to  higher  levels  of  understanding. 


36 


Research  on  Collaborative  Learning 


Literally  hundreds  of  studies  and  research  reviews  exist  and  most  demonstrate 
the  value  of  collaborative  learning.  For  example,  meta-analyses  comparing  the 
effects  of  small  group  and  individual  learning  environments  on  achievement  indicate 
effect  sizes  of  .75  (Johnson,  Maruyama,  Johnson,  Nelson,  &  Skon,  1981)  and  .63 
(Johnson  &  Johnson,  1989)  favoring  small  group  learning.  In  addition  to  improved 
learning,  numerous  other  studies  demonstrate  that  collaborative  learning  positively 
benefits  the  social-emotional  (positive  relationships  and  psychological  health) 
aspects  of  instruction  (Johnson  &  Johnson,  1992). 

The  cognitive  and  social  benefits  documented  for  traditional  classroom  use  of 
collaborative  learning  appear  to  transfer  to  computer-based  learning  environments  as 
well.  Some  studies  demonstrate  significantly  higher  achievement  for  collaborative 
groups  (Dalton  et  al.,  1989;  Hooper  et  ah,  1993;  Johnson  et  ah,  1985;  1986),  while 
others  find  no  significant  differences  between  individuals  and  small  groups  (Carrier 
&  Sales,  1987;  Makuch  et  ah,  1992;  Webb,  1985).  Webb  (1987)  reviewed  14  studies 
comparing  achievement  between  group  and  individual  computer  work  and  found 
similar  results;  only  five  studies  found  differences  favoring  the  group  activity. 
However,  Webb  concludes: 

The  important  result  is  that  no  study  found  greater  learning  among  students 
working  alone  than  students  working  in  groups.  This  suggests  that,  averaging 
over  all  students,  group  work  is  not  detrimental  to  students’  learning,  and  may 
be  beneficial,  (p.  195) 
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For  example,  groups  spend  more  time  interacting  on  task  (Johnson  et  al.,  1985)  and 
express  more  positive  attitudes  toward  working  at  computers  (Hooper  et  al,  1993). 
Although  variable,  these  results  mirror  the  positive  results  found  for  collaborative 
learning  in  the  non-computer  classroom.  Thus,  computer-based  collaborative 
learning  appears  to  be  an  enriching  experience  and  may  increase  achievement. 

Given  the  volume  of  studies  investigating  the  effects  of  computer-based 
collaborative  learning,  as  well  as  the  large  number  of  factors  involved  (age, 
experience  with  computers,  the  computer  activity,  achievement  measures,  etc.)  it  is 
no  wonder  that  results  vary.  This  disparity  of  results  notwithstanding,  the  general 
consensus  is  that  the  collaborative  use  of  classroom  computers  is  the  preferable 
implementation  strategy  (Watson,  1990).  One  reason  for  grouping  students  at 
computers  involves  solving  logistical  problems  (Hooper,  1 992);  in  many  instances, 
there  simply  are  more  students  than  computers.  A  second  and  more  educationally 
sound  reason  for  grouping  students,  rests  on  the  assumption  that  learning  occurs  in  a 
social  context  where  learners  interact  and  draw  on  one  another's  expertise  (Krajcik, 
Blumenfeld,  Marx,  &  Soloway,  1994).  In  the  latter  case,  grouping  students  at  the 
computer  is  based  on  the  roles  that  communication  and  interaction  play  in  creating 
meaning,  while  in  the  former  case,  grouping  students  may  be  based  solely  on 
classroom  expediency.  In  both  cases,  however,  teachers  expect  some  educational 
benefit  which  presumably  results  from  the  quality  of  communications  and  interactions 
that  occur  between  the  grouped  students. 
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In  summary,  considerable  evidence  exists  demonstrating  the  efficacy  of 
collaborative  learning,  both  in  a  computer-based  learning  environment  and  the  non¬ 
computer  classroom.  The  benefits  of  collaborative  learning  result  from  the  active 
engagement  of  students  in  an  equal  and  mutual  discourse  of  conversation  and 
communication  where  learners  make  connections  between  incoming  information  and 
between  the  incoming  information  and  prior  knowledge.  Clearly  collaborative 
learning  is  a  generative  activity. 

Despite  the  considerable  literature  devoted  to  computer-based  collaborative 
learning,  most  of  this  research  is  conducted  with  courseware  designed  for  individual 
use  (Jonassen,  1988).  Additionally,  computer-based  learning  is  dominated  by 
strategies  that  facilitate  only  superficial  levels  of  processing  (Hooper  &  Hannafin, 
1991).  These  types  of  software  may  be  inappropriate  for  group  use  (Hooper,  1992) 
and  may  explain  the  variability  of  results  found  for  computer-based  collaborative 
learning.  Several  researchers  (Hooper,  1992;  King,  1989;  Nastasi  &  Clements,  1992; 
Webb,  1987)  call  for  research  efforts  to  investigate  how  to  facilitate  and  strengthen 
the  use  of  effective  collaborative  processes.  It  is  my  contention  that  combining 
strategies  designed  to  engage  students  in  generative  learning  activities  should 
facilitate  student  interaction  and  increase  achievement. 
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Research  Questions 


Several  themes  emerge  from  the  literature  review  just  concluded.  First,  although 
limited  at  this  point,  research  supports  the  central  assertion  of  cognitive  flexibility 
theory  that  case-based  learning  produces  greater  knowledge  acquisition  and  transfer 
than  a  linear,  computer-based  approach  to  learning  the  same  material.  Could  student 
achievement  be  improved  by  modifying  a  CFT-based  computer  program  with 
additional  generative  strategies?  Additionally,  the  cognitive  flexibility  theory 
literature  only  evaluates  students  working  individually  in  the  CFT-based 
environment.  What  effect,  if  any,  would  pairing  students  have  on  the  effectiveness  of 
a  CFT-based  computer  microworld? 

Second,  research  demonstrates  that  generative  learning  strategies  are  effective 
for  increasing  comprehension  (Di  Vesta  &  Peverley,  1984;  Doctorow,  Wittrock,  & 
Marks,  1978;  Grabowski,  1996;  Johnsey,  Morrison,  &  Ross,  1992;  Linden  & 
Wittrock,  1981;  Stein  &  Bransford,  1979;  Wittrock,  1990;  1992;  Wittrock  & 
Alesandrini,  1990).  However,  most  generative  learning  studies  focus  on  learners 
working  alone.  Given  the  evidence  demonstrating  the  effectiveness  of  collaborative 
learning,  might  collaborating  pairs  generate  more  effective  explanations  and 
summaries  and  therefore,  reach  higher  levels  of  knowledge  acquisition  than  single 
students? 
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Finally,  a  common  theme  running  through  the  vast  literature  on  computer-based 

group  work  is  the  intense  focus  on  children. 

Energetic  computer-based  group  work  has  been  most  thoroughly  documented 
within  the  earliest  years  of  education...,  and  in  the  middle  and  secondary 
years.... There  is  very  little  relevant  research  on  this  topic  that  considers  further 
and  higher  education  or  work  in  training  communities.  (Crook,  1994,  p.  124) 

Given  the  relative  paucity  of  research  regarding  adult-oriented,  computer-based 

collaborative  learning,  is  collaborative  learning  effective  for  advanced  learners? 

Also,  can  the  collaborative  process,  and  hence  achievement,  be  improved  by 

integrating  the  opportunity  for  collaboration  with  other  generative  activities,  such  as 

specific  generative  learning  strategies  or  CFT-based  computer  learning 

environments?  This  study  was  designed  to  address  these  questions  by  assigning 

single  and  paired  students  to  work  on  two  qualitatively  different  CFT-based  computer 

programs  (a  base  program  and  a  program  enriched  with  additional  generative  learning 

strategies). 


Research  Hypotheses 

The  major  research  question  involved  investigating  the  effects  of  two  generative 
learning  activities  (generative  learning  theory  and  collaborative  learning)  on  the 
performance  of  advanced  learners  in  a  cognitive  flexibility-based  computer 
microworld.  Two  important  questions  emerge  concerning  the  integration  of  cognitive 
flexibility  theory,  generative  learning  theory,  and  collaborative  learning: 
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(1)  Would  the  inclusion  of  embedded  explanation  and  summary  cues 
(generative  learning  strategies)  result  in  differences  in  learning  in  a 
CFT-based  computer  microworld? 

(2)  How  would  the  addition  of  collaborative  learning  affect  learning  in  this 
environment? 

Based  on  the  literature,  it  was  hypothesized  that: 

(1)  Students  assigned  to  the  generative  program  would  out  perform  students 
assigned  to  the  base  program  on  the  posttest. 

(2)  Students  assigned  to  the  collaborative  condition  would  out  perform 
students  assigned  to  the  individual  condition  on  the  posttest. 

(3)  There  would  be  an  interaction  effect  between  the  version  of  program  and 
method  of  instruction  treatment  variables  as  measured  by  scores  on  the 
posttest.  A  synergy  would  develop  between  the  two  treatment  variables 
such  that  students  assigned  to  the  pair/generative  treatment  would  perform 
significantly  better  on  the  posttest  relative  to  the  other  treatment  groups. 
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CHAPTER  3 


METHODOLOGY 

Introduction 

This  study  used  a  computer-based  learning  program  to  provide  instruction  in  the 
complex  domain  of  transfusion  medicine.  This  program  integrated  multiple  cases 
with  access  to  relevant  information  sources  to  encourage  medical  students  to 
diagnose,  assess,  and  manage  infectious  and  non-infectious  adverse  events  of 
recipients,  and  donor-related  blood  transfusion  problems.  The  present  study 
investigated  the  relative  effects  of  complementary  generative  activities  (generative 
learning  strategies  and  collaboration)  on  the  knowledge  acquired  by  second-year 
medical  students  using  a  cognitive  flexibility  theory-based  computer  microworld. 

The  remainder  of  this  chapter  discusses  the  materials,  participants,  research  design, 
instrumentation,  and  instructional  procedures  used  in  this  study. 

Materials 

Cognitive  flexibility  theory  addresses  knowledge  acquisition  in  complex  and  ill- 
defined  subject  domains.  The  broad  field  of  medicine  is  considered  such  a  domain,  as 
are  the  specialty  fields  within  medicine  (Spiro  et  al.,  1988).  Transfusion  medicine  is: 
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That  area  of  medicine  which  includes  the  collection,  production,  storage  and 
administration  of  lymphohematopoietic  progenitors,  mature  blood  cells, 
plasma  and  plasma  components,  as  well  as  associated  services  to  be  used  in 
treatment  of  patients  with  specific  diseases.  (D.  Ambruso,  personal 
communication,  June  15, 1998) 

Transfusion  medicine  is  a  multidisciplinary  area  encompassing  a  number  of  basic 
sciences  (e.g.,  biochemistry,  cell  biology,  transplant  biology,  and  immunology), 
clinical  sciences  (e.g.,  hematology,  medicine,  surgery),  and  some  applied  sciences 
(D.  Ambruso,  personal  communication,  June  15, 1998).  Due  to  the  large  number  of 
interrelated  medical  and  science  domains,  no  prototypic  cases  exist.  Causes  and 
effects  interact  and  produce  a  variety  of  symptoms  or  require  a  number  of  different 
medical  interventions  that  depend  on  the  clinical  status  of  the  patient,  the  purpose  of 
the  transfusion,  and  from  whose  perspective  the  case  is  viewed  (Jonassen  et  al., 

1992).  Because  of  the  complexity  and  ill-structuredness  of  this  knowledge  domain,  a 
medical  school  professor  responsible  for  teaching  this  curriculum,  led  a  team  of 
physicians,  educational  consultants,  and  instructional  designers  (Transfusion 
Medicine  Team)  to  develop  “Handling  Transfusion  Hazards,”  a  computer  program 
based  upon  cognitive  flexibility  theory  principles. 

The  Program:  Handling  Transfusion  Hazards 

Diagnosing  and  treating  transfusion-related  problems  is  a  dynamic,  problem¬ 
solving  activity.  Clinicians  collect  information,  order  and  interpret  lab  tests,  and 
assess  and  manage  clinical  problems  in  their  patients,  including  defining  the 
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indication  for  use  of  blood  transfusions.  “Handling  Transfusion  Hazards”  instantiates 
this  process  by  providing  medical  students  with  realistic  cases  to  solve,  as  well  as  the 
tools  necessary  to  solve  the  cases. 

The  program  consists  of  four  distinct  parts:  (a)  program  introduction, 

(b)  pretest,  (c)  learning  phase,  and  (d)  posttest  (see  “Instruments”  this  chapter  for 
descriptions  of  the  pretest  and  posttest).  The  introduction  provides  information  to 
help  the  students  successfully  complete  the  program.  This  orientation  (a)  discusses 
the  program  content  and  learning  objectives,  (b)  provides  a  concept  map  (i.e., 
diagram)  that  illustrates  and  explains  each  part  of  the  program,  (c)  and  informs  the 
students  of  the  requirement  to  complete  the  program  in  its  entirety  by  sequencing 
through  the  pretest,  practice  cases,  and  posttest. 

The  learning  phase  contains  six  practice  cases  (scenarios)  that  require  the 
subjects  to  order  laboratory  tests  (if  appropriate)  to  help  them  form  a  diagnosis  and 
then  select  suitable  assessment  and  management  options.  The  cases  require  the 
students  to  access  relevant  factual  and  conceptual  information  which  is  then  applied 
to  solve  the  transfusion  problems  at  hand.  Each  case  contains  (a)  a  description  of  the 
case,  (b)  a  research  section,  (c)  an  actions  section,  and  (d)  a  help  section  (Figure  3.1 
presents  a  sample  case  screen).  These  sections  are  described  below. 
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Figure  3.1.  Sample  Case  Screen  from  Practice  Case  Three 


It’s  Saturday  morning,  July  5,  and  you  receive  a  frantic 
page  from  the  drawing  center  at  your  hospital  blood  bank. 

Your  hospital  routinely  draws  volunteer  blood  donors  for 
processing  by  the  local  community  blood  center  and  the 
intern  on  duty  provides  medical  coverage. 

A  donor  has  just  fainted  and  your  help  is  required 
immediately.  You  breathlessly  approach  the  donor  room 
to  find  a  pale,  sweaty  donor  lying  on  a  cot. 
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•  Case  summary:  Upon  selecting  a  case,  students  are  presented  with  the 
case  summary  which  provides  basic  contextual  information  (see  Fig.  3.1). 

•  Research:  The  research  section  contains  thematic  information  sources  that 
provide  structural  information  relevant  to  the  case. 
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•  History/physical:  This  section  provides  the  basic  patient 

background  information  necessary  for  the  learners  to  begin  the 
diagnostic  process.  Figure  3.2  presents  a  sample  history/physical 
screen. 


Figure  3.2,  Sample  History  Screen  from  Practice  Case  Three 
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•  Perspectives:  Each  case  contains  opinions  from  several  significant 
operatives,  such  as  the  attending  physician,  resident,  phlebotomist, 
blood  bank  director,  surgeon,  pediatrician,  internist,  fellow,  donor, 
recipient,  gastroenterologist,  and  patient.  Usually  four  or  five 
perspectives  are  available  based  on  the  relevance  of  the 
information  to  the  case  content.  Figure  3.3  presents  a  sample 
perspectives  screen. 

Figure  3.3.  Sample  Perspectives  Screen  from  Practice  Case  Three 


•  Similar  cases:  The  program  contains  a  bank  of  24  cases  that  are 
less  detailed  than  the  primary  learning  cases.  These  cases  provide 
information  pertinent  to  the  selected  practice  case.  The  similar 
cases  menu  provides  two  choices:  closely  related  similar  cases  and 
other  suggested  similar  cases.  Closely  related  similar  cases  share 
the  following  attributes  with  the  practice  case:  group  involved, 
type  of  risk,  pathophysiology,  symptoms,  and  screening  tests; 
whereas,  the  other  suggested  similar  cases  share  only  one  or  two  of 
the  attributes  above.  Figure  3.4  shows  a  sample  similar  cases 


screen. 


Figure  3.4.  Sample  Similar  Cases  screen  from  Practice  Case  Three 


•  Help:  The  help  section  provides  three  choices:  textbook,  quit,  and 
map. 

•  Textbook:  The  program  contains  a  transfusion  medicine  textbook 
that  provides  detailed  information  on  donor  and  recipient 
transfusion  issues,  as  well  as  access  to  a  glossary  of  medical 
terminology.  Figure  3.5  presents  the  textbook  index  screen. 


Figure  3.5.  Textbook  Index  Screen 


•  Quit:  The  quit  function  allows  the  students  to  exit  the  program.  If 
selected  this  option  saves  the  student’s  work  up  to,  but  not 
including  the  current  case. 
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•  Map:  This  function  takes  the  students  to  the  conceptual  map 
contained  in  the  program  orientation. 

The  research  resources  and  textbook  contain  the  declarative  (factual)  and 
conceptual  knowledge  required  to  support  the  students’  problem-solving  processes;  as 
such,  they  remain  available  throughout  the  diagnosis  and  treatment  phases  (labs, 
assessment,  and  management)  which  are  described  below. 

•  Actions:  Each  case  requires  the  students  to  order  lab  tests  (if  appropriate), 
assess,  and  manage  a  transfusion  medicine  problem. 

•  Laboratory  tests:  Students  have  the  option  of  ordering  many 
different  types  of  laboratory  tests  to  help  them  develop  and 
confirm  their  diagnosis.  Upon  selecting  the  lab  test  portion  of  the 
program,  students  are  given  the  option  to  choose  from  five 
different  types  of  lab  tests:  chemistry,  hematology/coagulation, 
blood  bank/serology,  microbiology/urinalysis,  and  diagnostic 
procedures/radiology.  Each  of  these  test  types  contain  between  4 
and  20  tests  and  procedures.  Upon  selecting  a  test,  students 
receive  immediate  test  results  (a  test  value)  and  feedback 
concerning  the  appropriateness  of  the  test.  After  ordering  all  of  the 
tests  for  a  given  type  (e.g.,  chemistry),  the  students  have  the  option 
to  view  a  list  of  suggested  tests. 
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•  Assessment  and  Management:  Assessment  and  management 
actions  proceed  in  the  same  fashion.  Students  choose  their 
courses  of  action  and  receive  immediate  feedback  concerning  the 
advisability  of  each  action  taken.  Then  they  have  the  option  to 
view  the  correct  actions.  After  the  students  complete  the 
management  actions,  each  case  ends  with  a  summary  of  the 
learning  outcomes  specific  to  that  case. 

“Handling  Transfusion  Hazards”  facilitates  knowledge  acquisition  in  the 
complex  domain  of  transfusion  medicine  by  giving  students  practice  solving  actual 
clinical  problems.  This  program  achieves  its  objectives  by  combining  case-based 
instruction  with  the  structural  support  of  thematic  information  resources  (patient’s 
history/physical,  a  transfusion  medicine  textbook,  perspectives  of  several  case¬ 
relevant  operatives,  and  other  prototypically  similar  cases).  Specifically,  this 
program: 

1 .  avoids  oversimplifying  instruction  since  no  single  case  is  prototypic, 

2.  provides  multiple  representations  through  the  varied  nature  of  the  cases 
and  different  sources  of  information  available  for  processing  the  cases, 

3.  supports  context-dependent  knowledge  acquisition  by  providing  real  life 
cases,  as  well  as  realistic  information  sources,  and 

4.  emphasizes  knowledge  construction  through  the  analytical  processes  built 
into  the  program.  Students  must  access  different  sources  of  declarative 
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and  conceptual  knowledge  to  solve  the  cases.  In  the  process  the  students 
build  complex  schemata  consisting  of  procedural  (how  to)  knowledge 
rather  than  an  assortment  of  unrelated  facts  (Jonassen  et  al.,  1992). 

In  summary,  “Handling  Transfusion  Hazards”  fosters  contextualized  reasoning 
(Resnick,  1987).  The  program  provides  medical  students  with  the  opportunity  to 
scientifically  reason  and  solve  real-world  medical  problems  just  as  practicing 
clinicians  would,  and  therefore,  is  clearly  a  generative  activity  (see  Appendix  A  for  a 
copy  of  a  practice  case). 

Program  Development  and  Evaluation 

An  extensive  development  and  formative  evaluation  process  was  used  to  develop 
“Handling  Transfusion  Hazards.”  This  process  took  almost  two  years  and  is  detailed 
below. 

•  First,  the  content  topics  in  transfusion  medicine  were  identified  by  the 
physicians  responsible  for  teaching  this  curriculum  and  organized  under 
the  theme  of  risk  management  to  donors  and  recipients.  Selection  of  these 
topics  was  guided  by  learning  objectives  established  for  the  transfusion 
medicine  curriculum. 

•  Second,  this  information  was  organized  into  a  database  by  the  Transfusion 
Medicine  Team  according  to  adverse  events  (for  example,  Hepatitis  A, 
Hepatitis  B,  HIV,  etc.).  Cognitive  flexibility  theory  was  selected  as  the 
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theoretical  framework  for  presenting  this  information  in  a  computerized, 
case-based  format  for  two  reasons:  (a)  the  learning  objectives  involve 
application  and  transfer  of  information  and  (b)  cognitive  flexibility  theory 
provides  the  means  to  create  an  authentic  environment  that  promotes 
critical  thinking  and  problem  solving  (additional  educational  outcomes  the 
Transfusion  Medicine  Team  wished  to  facilitate). 

•  Third,  the  practice  and  test  cases  were  developed  based  upon  actual 
clinical  cases.  Issues  considered  at  this  point  included  the  number  of  cases 
to  use,  the  appropriate  resources  (textbook,  etc.)  necessary  for  the  students 
to  solve  the  cases,  and  the  relationships  between  the  cases. 

•  Fourth,  a  complete  working  program  was  developed  leading  to  formative 
evaluation  of  the  program  by  users  (medical  students,  residents,  and 
fellows)  and  expert  review  of  content.  Formative  evaluation  was 
conducted  on  individual  practice  cases  and  the  total  program  to  check  for 
proper  operation  of  the  program.  Content  review  was  conducted  by  two 
transfusion  medicine  experts  (not  connected  with  the  project)  who 
checked  the  content  and  operation  of  the  program  for  accuracy  and 
completeness. 

•  Finally,  changes  suggested  by  the  expert  reviewers  were  incorporated. 

This  study  used  two  versions  of  this  program:  the  version  described  above  (base) 

and  a  generative  version.  The  generative  program  includes  embedded  generative 
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learning  strategies  (explanation  and  summarization  cues)  that  require  the  students  to 
explain  their  decisions  and  to  summarize  the  main  concepts  of  each  transfusion 
medicine  case.  Providing  explanations  enables  students  to  reorganize  and  clarify 
their  ideas  which  helps  them  recognize  gaps  in  understanding  and  increase  learning 
(Webb  &  Lewis,  1988).  Summarization  involves  paraphrasing  the  information  to  be 
learned  and  through  further  encoding,  helps  students  to  consolidate  and  strengthen 
what  was  learned  and  provide  feedback  concerning  the  degree  of  understanding 
achieved  (Hooper,  1992;  Yager,  et  al.,  1985).  These  strategies  should  help  the 
students  make  the  connections  between  the  multiple  contexts  in  which  concepts  apply 
leading  to  greater  knowledge  acquisition. 

The  generative  program  version  requires  students  to  justify  (explain)  their 
decisions  when  ordering  lab  tests,  and  assessing  and  managing  each  case.  After 
explaining  their  actions,  the  students  can  view  the  results,  corrective  feedback,  and 
the  correct  actions  just  as  they  would  in  the  base  version.  Finally,  before  proceeding 
to  the  next  case,  the  program  prompts  the  students  to  summarize  the  main  concepts 
contained  in  the  case,  and  to  highlight  any  similarities  or  differences  with  other  cases 
(Appendix  B  contains  examples  of  the  generative  version  screens). 

The  base  program  was  modified  to  produce  the  generative  program  version  by 
the  author  with  the  help  of  a  computer  programmer  affiliated  with  the  Transfusion 
Medicine  Team.  The  introductory  information,  instructional  content,  and  lesson 
sequencing  remained  the  same.  The  only  changes  involved  embedding  the 
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explanation  and  summary  cues  and  providing  the  screen  space  necessary  to  respond 
to  the  cues.  Since  the  changes  encompassed  how  the  program  functions,  and  not 
program  content,  the  author  conducted  an  operational  evaluation  to  ensure  the  reliable 
operation  of  the  generative  program. 


Participants 

One  hundred  and  thirty-two  second-year  medical  students,  70  men  and  62 
women,  participated  in  this  study.  These  subjects  were  enrolled  in  Pathophysiology 
of  Disease,  a  required  course  for  all  students  attending  a  major  medical  university 
located  in  a  large  western  city.  The  subjects  completed  “Handling  Transfusion 
Hazards”  during  the  three-week  hematology  rotation  of  the  Pathophysiology  of 
Disease  course.  All  subjects  completed  the  program;  however,  thirty-one  students 
were  excluded  from  the  final  analysis  due  to  missing  or  inaccurate  data  and 
noncompliance  with  study  conditions  (see  Chapter  Four,  Data  Collection  and 
Analysis  Screening  Procedures).  Thus,  one  hundred  and  one  students  were  included 
in  final  statistical  analysis. 


Assignment  to  Treatment 

Effective  interaction  stimulates  “mindful  learning”  (the  employment  of 
nonautomatic  volitional  and  metacognitively  guided  processes  of  learning)  resulting 
in  greater  cognitive  effort  and  deeper  processing  of  information  (Salomon,  1985; 
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Salomon  &  Globerson,  1987;  1988).  Interdependence  must  be  established  for 
students  to  interact  effectively  (Johnson  &  Johnson,  1992);  however,  simply  placing 
students  together  and  asking  that  they  work  as  a  team  may  not  establish  the  level  of 
interdependence  required  for  a  good  collaborative  effort.  Salomon  and  Globerson 
(1988)  write: 

A  team  is  a  social  system,  and  as  such  it  is  a  qualitatively  different  entity  than 
a  few  individuals  working  alone  side-by-side.  Behaviors  and  cognitions  in  the 
group  have  two  major  characteristics:  they  become  interdependent  and  this 
interdependence  develops  over  time  in  a  reciprocal  manner.  This  developing 
interdependence  implies  that  individuals’  cognitive  processes  affect  and 
become  affected  by  the  ones  of  the  other  team  members.  Communication 
among  the  team  members  serves  as  the  means  for  the  gradually  growing 
interdependence  of  cognitions  and  behaviors  such  that  efforts  (or  effort 
avoidance)  become  coordinated  and  shared,  (p.  93) 

Interdependence  is  likely  to  be  low  during  short  research  experiments  when 
subjects  have  little  previous  interpersonal  knowledge  or  interaction  with  one  another 
(Hooper,  Sales,  &  Rysavy,  1994;  Salomon  &  Globerson,  1988).  Low 
interdependence  may  explain  the  variability  of  results  concerning  achievement  in  the 
computer-based  collaborative  learning  literature,  as  well  as  the  unanticipated  results 
in  the  pilot  study  (described  later  in  this  chapter).  Therefore,  paired  subjects  picked 
their  partners.  The  assignment  process  proceeded  as  follows: 

1 .  The  students  were  previously  assigned  to  one  of  twelve  unit  teaching  labs 
(UTLs)  ranging  from  8  to  12  students  each,  therefore,  the  method  of 
instruction  variable  (single  or  pairs)  was  randomly  assigned  to  the  UTLs. 
One  UTL  was  assigned  to  the  single  condition  for  every  two  UTLs 
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assigned  to  the  paired  condition,  resulting  in  the  assignment  of  4  UTLs  (44 
students)  to  the  single  condition  and  8  UTLs  (88  students)  to  the  paired 
condition. 

2.  A  program  version  was  then  randomly  assigned  to  each  individual 
student  or  pair. 

This  process  created  four  treatment  groups  (see  Figure  3.6).  Groups  1  and  2 
were  each  comprised  of  22  students  working  individually.  These  groups  differed  by 
the  version  of  program.  Groups  3  and  4  were  each  comprised  of  22  collaborative 
pairs.  Again  these  groups  differed  by  the  program  version.  The  breakdown  of 
students  and  treatment  groups  for  the  final  analysis  consisted  of  31  single  students  (18 
students  assigned  to  Group  1  and  13  students  assigned  to  Group  2)  and  35 
collaborative  pairs  (20  pairs  assigned  to  Group  3  and  15  pairs  assigned  Group  4). 

Design 

This  study  used  a  2  X  2  factorial  design  (see  Figure  3.6)  to  address  the  question 
of  how  embedded  generative  learning  strategies  and  collaboration  affect  knowledge 
acquisition  among  advanced  learners  (medical  students)  in  a  CFT-based  computer 
microworld. 
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Figure  3.6.  Experimental  Design 


Program  Version 

Instructional  Method 

Base 

Generative 

Single 

Group  1 

Group  2 

Pair 

Group  3 

Group  4 

The  experimental  design  employed  the  pretest-posttest  control  group  design  espoused 
by  Campbell  &  Stanley  (1963)  and  is  graphically  depicted  below: 

R  O  Xi  O 
R  0  X2  O 
R  O  X3  O 
R  O  X4  O. 


Variables 


Independent  Variables 

Two  independent  variables  were  studied  (see  Figure  3.6).  The  first  independent 
variable  was  the  version  of  computer  program.  As  previously  discussed,  a  base 
computer  program  was  developed  according  to  cognitive  flexibility  theory  principles. 
A  generative  version  was  then  developed  by  embedding  two  generative  learning 
strategies  into  the  base  version.  Creating  two  similar,  yet  qualitatively  different 
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programs  permitted  testing  the  effect  generative  learning  strategies  exert  on  learning 
in  a  CFT-based  computer  microworld.  The  second  independent  variable  consisted  of 
the  instructional  method  (individual  versus  collaborative  learning),  which  permitted 
testing  the  effect  of  collaboration  on  the  learning  among  advanced  learners  in  a 
cognitive  flexibility  environment. 

Dependent  Variable 

The  dependent  variable  was  the  knowledge  acquired  (constructed)  by  the 
subjects  as  measured  by  a  posttest  taken  at  the  conclusion  of  the  learning  phase  of  the 
computer  program.  The  dependent  measure  consisted  of  three  cases  that  required  the 
subjects  to  diagnose  and  treat  transfusion  related  problems  similar  to  the  problems 
encountered  in  the  practice  cases.  Each  posttest  case  measured  the  students’  mastery 
of  the  facts  and  ability  to  think  through  and  solve  the  transfusion  medicine  problems 
encountered.  The  test  cases  were  similar  in  structure  to  the  practice  cases  except  that 
the  information  resources  (e.g.,  perspectives,  similar  cases,  and  textbook)  were 
removed  (see  Figure  3.7). 
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Figure  3.7.  Sample  Screen  from  Test  Case  B 


As  discussed  in  Chapter  One,  collaborative  learning  studies  typically  require 
subjects  assigned  to  the  group  condition  to  test  independently  to  measure  the 
“individual”  knowledge  acquired  as  a  result  of  collaborating.  Collaborative  learning 
is  premised  on  the  notion  that  “two  heads  are  better  than  one,”  where  the  level  of 
group  generated  knowledge  is  greater  than  the  knowledge  of  the  individual 
participants  (Whipple,  1987).  Also,  the  research  demonstrates  that  group-to- 
individual  transfer  occurs  at  least  to  the  same  degree  between  subjects  assigned  to 
collaborate  and  subjects  assigned  to  the  single  condition.  Therefore,  in  this  study,  the 
subjects  assigned  to  the  collaborative  condition  tested  with  their  partners.  This 
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permitted  examining  individual  versus  socially  constructed  knowledge.  The 
instruments  section  provides  a  more  detailed  description  of  the  posttest. 

Instruments 


Pretest  and  Posttest 

The  pretest  and  posttest  measured  the  subjects’  understanding  of  the  transfusion 
medicine  knowledge  domain.  The  students’  scores  on  the  pretest  and  posttest  were 
based  on  the  lab  tests  they  ordered,  and  how  they  assessed  and  managed  three  cases 
conceptually  similar  to  the  cases  presented  during  the  learning  phase  of  the  program. 
The  point  system  used  to  score  the  learners’  performance  is  outlined  below: 

•  Lab  tests:  Each  lab  test  ordered  was  scored  on  a  scale  from  -1  to  +2 
depending  on  how  essential  it  was  for  establishing  a  diagnosis  for  the  case, 
its  affect  on  the  patient,  and  cost  considerations. 

•  Assessment:  Each  action  selected  for  assessing  a  case  received  a  score  of 
-1  or  +1  depending  on  the  validity  and  appropriateness  of  the  selection. 

•  Management:  Each  management  action  received  a  score  of -1  or  +1 
depending  on  the  appropriateness  of  the  selection  for  the  case. 

The  three  test  cases  contained  a  total  of  172  lab  tests,  19  assessment,  and  16 
management  choices  (see  Appendix  C  for  an  example  test  case).  There  were  30  lab 
test  points,  9  assessment  points,  and  8  management  points  for  a  total  of  47  points. 
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The  same  cases  were  used  for  the  pretest  and  posttest  (see  the  pilot  study  for 
justification). 

The  test  cases  were  developed  in  conjunction  with  the  learning  cases.  Two 
transfusion  medicine  experts  reviewed  the  test  cases  for  content  validity, 
comprehensiveness,  and  accuracy  as  part  of  the  content  review  process.  They  were 
asked  to  judge  the  match  between  the  learning  objectives,  the  practice  cases,  and  the 
test  cases,  as  well  as  to  evaluate  the  rationale  used  to  develop  the  grading  scales.  The 
experts  were  asked  to  arrive  at  consensus  regarding  these  issues.  Changes  were  made 
to  the  test  cases  based  upon  the  experts’  feedback. 

Procedure 


Pilot  Study 

The  base  computer  program  was  piloted  during  the  spring  of  1997  to  determine: 

1 .  If  differences  in  learning  exist  between  single  and  paired  users, 

2.  If  medical  students  using  a  computer-based  learning  program  would 
effectively  collaborate,  and 

3.  If  the  test  cases  were  parallel  forms. 

The  participants  in  the  pilot  study  were  second-year  medical  students,  one  year 
further  advanced  in  medical  school  than  the  current  subjects.  All  students 
participated,  resulting  in  a  subject  population  of  one  hundred  and  forty-two. 
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To  answer  these  questions  the  base  program  was  modified  slightly.  Since  the 
program  consisted  often  cases,  the  decision  was  made  to  drop  one  non-infectious 
disease  case  to  produce  a  module  containing  two  pretest  cases,  five  learning  cases, 
and  two  posttest  cases.  The  pretest  and  posttest  each  contained  one  infectious  disease 
case  and  one  blood  donor  case,  while  the  learning  phase  contained  infectious,  non- 
infectious,  and  donor  cases.  Two  versions  of  the  base  program  were  developed, 
differing  by  alternate  forms  of  the  pretest  and  posttest.  In  other  words,  the  pretest  for 
program  version  A  was  the  posttest  for  program  version  B  and  vice  versa. 

The  pilot  study  was  conducted  between  April  2, 1997  and  May  22, 1997. 
Pretest/posttest  results  indicated  that  the  test  versions  were  not  parallel.  The  means 
for  one  infectious  disease  test  case  (test  case  2)  were  higher  than  the  means  for  the 
second  infectious  disease  test  case  (test  case  1).  A  series  of  2  (Grouping)  X  2 
(V ersion)  ANOVAs  were  run  to  determine  if  a  lack  of  parallelism  existed.  The 
ANOVAs  resulted  in  several  significant  main  effects  and  interactions  by  program 
version  for  the  recipient  cases.  The  data  suggest  that  test  case  2  was  easier,  which 
corresponded  with  the  higher  mean  scores  on  this  test. 

Although  the  tests  were  not  equivalent  versions,  it  was  still  possible  to  determine 
if  differences  in  learning  existed  between  singles  and  pairs.  In  this  case  two  separate 
ANCOVAs  were  run  (by  program  version)  comparing  the  means  for  singles  and  pairs 
for  program  version  A  and  comparing  the  means  for  program  version  B.  This 
analysis  indicated  that  no  significant  differences  existed  between  the  single  and 
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paired  students  using  the  same  versions  of  the  program.  Although  not  anticipated, 
this  result  does  not  deviate  from  the  collaborative,  computer-based  learning  literature, 
where  some  studies  demonstrate  higher  achievement  for  collaborative  groups  versus 
individuals,  while  others  find  no  significant  differences. 

Finally,  to  determine  if  program  users  would  effectively  collaborate,  five 
volunteer  pairs  were  videotaped  using  the  program.  A  coding  scheme  containing  two 
main  interaction  types  (on-task  and  off-task)  and  four  verbal  interaction  categories 
(questions  and  statements  for  both  interaction  types)  was  developed  as  an  observation 
checklist.  The  four  interaction  categories  were  further  subdivided  into  two 
categories:  (a)  suggestions,  opinions,  directions,  and  (b)  explanations,  evaluations. 
The  coding  scheme  also  contained  a  provision  for  categorizing  an  interaction  as  either 
cognitive  conflict  or  co-construction. 

Verbal  interactions  were  coded  in  terms  of  their  frequency  of  occurrence  by  two 
independent  coders.  To  determine  if  cognitive  conflict  or  co-construction  existed,  the 
coders  were  asked  to  document  cases  of  cognitive  conflict;  if  cognitive  conflict  was 
not  present,  it  was  assumed  that  co-construction  existed.  The  coders  documented  the 
verbal  interactions  for  the  practice  and  posttest  phases  of  the  program.  Average  inter¬ 
coder  agreement  was  .95.  Although  the  sample  was  small,  the  data  suggest  that  the 
medical  students  remained  highly  task  focused  during  the  two  program  phases 
observed.  Furthermore,  the  interaction  patterns  demonstrate  that  the  students 
concentrated  their  efforts  in  giving  and  receiving  explanations,  which  as  previously 
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discussed,  are  positive  collaborative  behaviors.  Finally,  the  coders  failed  to  document 
any  cases  of  cognitive  conflict,  suggesting  that  these  pairs  created  a  co-constructive, 
collaborative  environment. 

The  results  of  the  pilot  study  led  to  the  questions  of  interest  in  the  present  study, 
as  well  as  the  following  procedural  changes: 

1 .  Creation  of  a  second  program  (i.e.,  the  generative  version). 

2.  Since  the  test  cases  were  demonstrated  not  to  be  parallel  forms  of  the 
measure,  the  decision  was  made  to  drop  one  test  case  and  use  the  same 
cases  for  both  the  pretest  and  posttest. 

3.  Adding  the  dropped  test  case  to  the  bank  of  learning  cases  to  provide 
comprehensive  coverage  of  the  transfusion  medicine  domain. 

4.  Assigning  the  instructional  method  condition  to  the  UTLs  rather  than  to 
each  individual  student. 

Instructional  Intervention 

The  Pathophysiology  of  Disease  course  was  broken  into  three  sub-courses.  The 
transfusion  medicine  curriculum  was  taught  during  the  hematology  block  of 
instruction.  During  a  30-minute  lecture  on  the  first  day  of  the  hematology  course,  the 
students  were  introduced  to  the  computer  module.  The  transfusion  medicine 
curriculum  manager  discussed  the  program  content  and  learning  objectives,  the 
logistics  involved  to  obtain  a  computer  disk,  and  the  requirement  to  complete  the 
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program  and  grading  criteria.  The  students  were  also  told  that  they  must  complete  the 
program  as  assigned  (i.e.,  if  assigned  to  work  individually,  they  must  not  consult  with 
any  other  students;  and  if  assigned  to  work  collaboratively,  they  must  complete  the 
program  only  with  their  partner).  Finally,  the  students  were  directed  not  to  discuss 
program  content  or  operation  with  any  other  students  enrolled  in  Pathophysiology  of 
Disease.  Appendix  D  contains  a  copy  of  these  instructions. 

The  program  was  loaded  on  six  computers  in  the  medical  school's  Learning 
Resources  Center  (LRC).  Three  computers  were  designated  as  the  primary  computers 
for  running  the  program;  these  computers  were  physically  separated  from  one 
another.  The  other  three  computers  containing  the  program  were  co-located  and  were 
considered  backups.  Each  individual  student  and  collaborative  pair  were  issued  a 
diskette  that  provided  access  to  the  program  and  recorded  their  actions.  Due  to  the 
medical  students’  tight  schedules  and  the  limited  number  of  computers,  the  students 
were  given  approximately  three  weeks  (March  23, 1998  to  April  10, 1998)  to 
complete  the  program.  The  medical  school  relies  heavily  on  the  honor  system, 
however  to  monitor  compliance  with  the  curriculum  manager’s  instructions  to  work 
individually  or  in  pairs  as  assigned,  the  author  was  present  while  22  single  students 
and  31  pairs  worked  through  the  program.  After  completing  the  program  the  students 
turned  their  disks  into  the  LRC  staff  for  collection. 
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CHAPTER  4 


RESULTS 

Introduction 

This  study  examined  the  relative  effects  of  embedded  generative  strategies  and 
collaboration  on  knowledge  acquired  by  advanced  learners  in  a  cognitive  flexibility 
theory-based  computer  microworld.  The  independent  variables  were  program  version 
(base  or  generative)  and  method  of  instruction  (single  or  pairs).  The  dependent 
variable  was  the  score  achieved  on  a  posttest  consisting  of  three  transfusion  medicine 
cases.  Analysis  of  covariance  (ANCOVA)  was  used  to  test  the  experimental 
hypotheses.  In  addition,  multivariate  analysis  of  variance  (MANOVA),  analysis  of 
variance  (ANOVA),  and  t-test  statistical  techniques  were  used  in  a  secondary  analysis 
of  the  data.  This  chapter  presents  the  results  of  these  analyses  and  is  organized  as 
follows: 

1 .  Data  collection  and  screening  procedures. 

2.  Primary  data  analyses. 

(1)  Compliance  with  the  assumptions  for  ANCOVA, 

(2)  Summary  and  descriptive  statistics,  and 

(3)  Analysis  of  covariance. 
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3.  Secondary  data  analyses. 

(1)  Multivariate  analysis  of  variance, 

(2)  Program  effectiveness, 

(3)  Time  spent  on  instruction, 

(4)  Analysis  of  explanations  and  summaries,  and 

(5)  Analysis  of  student  comments. 

4.  Summary  of  results. 

Data  Collection  and  Analysis  Screening  Procedures 
The  students’  disks  recorded  pretest  and  posttest  results  (selections  and  scores), 
as  well  as  other  interactional  data  (e.g.,  time  information,  student  explanations  and 
summaries,  and  program  critiques).  These  data  were  extracted  from  the  disks, 
checked  for  accuracy,  and  hand  entered  into  the  Statistical  Package  for  the  Social 
Sciences  (SPSS  7.5  for  Windows,  1996)  data  editor.  This  database  served  as  the 
master  file  for  the  primary  and  secondary  statistical  analyses. 

As  previously  discussed,  medical  students  enrolled  in  the  hematology  rotation  of 
the  Pathophysiology  of  Disease  course  were  required  to  complete  this  program.  One 
hundred  and  thirty-two  students  completed  the  program;  however,  thirty-one  students 
were  eliminated  from  the  final  analysis  due  to  inaccurate  data  or  noncompliance. 
Three  students  were  dropped  because  of  extreme  scores  or  missing  data.  Two 
students  assigned  to  the  single  condition  were  excluded  because  they  violated  the 
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requirement  to  work  independently.  The  remaining  twenty-six  students  were 
eliminated  due  to  lack  of  effort.  These  students  either  failed  to  provide  the 
explanations  and  summaries  required  of  the  generative  version  of  the  program,  or 
they  chose  to  spend  less  than  thirty-six  minutes  (the  established  minimal  time 
necessary  to  complete  the  program)  on  the  six  practice  cases.  Thus,  the  final  analysis 
included  one  hundred  and  one  students  (31  singles  and  35  pairs). 

Primary  Data  Analyses 

This  study  used  a  2  X  2  factorial  design  to  test  for  the  main  effects  and 
interaction  predicted  in  Chapter  Two.  The  students  were  randomly  assigned  to  the 
different  levels  of  the  independent  variables.  An  analysis  of  variance  on  the  subjects’ 
pretest  scores  suggests  that  randomization  was  achieved  (see  discussion  of  random 
assignment  in  the  next  section);  however,  analysis  of  covariance  (using  the  pretest  as 
the  covariate)  was  selected  to  conduct  the  primary  data  analysis  to  increase  the  power 
of  the  study.  This  section  discusses  the  assumptions  of  analysis  of  covariance 
(ANCOVA),  provides  summary  and  descriptive  data,  and  concludes  with  a 
presentation  of  the  ANCOVA  results. 

Assumptions  for  Analysis  of  Covariance 

Random  and  Independent  Errors.  Each  subject  was  randomly  assigned  to  one 
treatment  group  according  to  the  process  described  in  Chapter  Three.  An  analysis  of 
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variance  conducted  on  the  pretest  for  the  four  treatment  groups  suggests  that  random 
assignment  was  achieved:  F(3, 63)=.413,  p=.744.  Additionally,  the  researcher 
observed  84  students  running  the  computer  program  and  noticed  only  two  students 
not  complying  with  the  requirement  to  work  as  assigned  (i.e.,  individually  or  pairs). 
These  students  were  excluded  from  the  statistical  analyses.  The  students’  disks  also 
recorded  the  dates  and  times  the  students  accessed  the  program.  Analysis  of  the  dates 
and  times  of  the  students  not  observed  suggest  compliance  with  the  requirement  to 
work  as  assigned.  To  gauge  the  level  of  compliance  with  the  condition  to  refrain 
from  discussing  the  program,  the  researcher  randomly  asked  students  what  they  had 
heard  about  the  program  prior  to  arriving  at  the  LRC.  Most  of  the  students  reported 
hearing  that  the  program  was  “long”  or  “frustrating,”  but  none  of  the  students 
reported  any  discussion  related  to  the  program’s  content.  An  analysis  of  variance 
conducted  on  pretest  data  (F(2,  63)=.217,  p=.805)  and  posttest  data 
(F(2,  63)=.204,  p=.816),  by  the  week  the  program  was  completed,  suggests  this 
noncompliance  had  a  negligible  effect  on  the  students’  scores.  Finally,  inspection  of 
a  residual  plot  of  posttest  scores  (see  Appendix  F)  suggests  that  the  assumption  of 
random  and  independent  errors  was  met. 

Normality.  Analysis  of  covariance  is  robust  to  departures  of  the  sample  from 
normality  (Lomax,  1992);  nevertheless,  the  subjects’  posttest  scores  were  analyzed  to 
determine  if  the  requirement  of  normality  was  met.  Mean  posttest  scores  were  22.9 
with  a  standard  deviation  of  5.18.  The  skewness  statistic  (-.270)  indicates  that  the 
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distribution  was  slightly  skewed  to  the  left.  The  kurtosis  statistic  (-.469)  indicates 
that  the  center  and  tails  of  the  distribution  were  slightly  shorter  than  that  of  a  normal 
distribution.  The  ratio  of  each  of  these  statistics  to  their  standard  error  can  be  used  as 
a  test  of  normality  (SPSS,  1996).  The  ratios  of  -.92  (skewness)  and  -.81  (kurtosis) 
fall  between  the  values  -2  and  +2,  and  the  shape  of  the  histogram  is  fairly  symmetric, 
therefore  the  posttest  scores  approximate  a  normal  distribution  curve.  Stem  and  leaf 
plot  analysis  of  posttest  scores  and  the  Kolmogorov-Smimov  test  of  normality 
(F  (66)=.  102,  p=.083)  confirm  that  the  posttest  scores  approximate  a  normal 
distribution  (see  Appendix  F). 

Homogeneity  of  Variance.  Levene’s  test  of  equal  variances,  and  Cochran’s  C, 
and  Bartlett-Box  F  tests  of  homogeneity  of  variance  were  conducted  on  the  posttest 
data  to  determine  if  the  cell  variances  were  equal.  Levene’s  F(3,  62)=.697,  p=.558, 
Cochran’s  C(16,4)=.282,  p=1.000,  and  Bartlett-Box  F(3, 6475)=.069,  p=.977  all 
support  the  null  hypothesis  that  the  population  cell  variances  were  equal. 

Linearity  and  Homogeneity  of  Regression  Slopes.  Inspection  of  the 
scattergrams  showing  the  relationship  between  the  pretest  and  posttest  scores  for  the 
entire  sample,  as  well  as  each  treatment  group,  support  the  assumption  of  linearity.  In 
each  case  a  straight-line  best  fit  the  data  (see  Appendix  F). 

The  assumption  of  homogeneity  of  regression  slopes  requires  the  slopes  of  the 
regression  lines  to  be  the  same  for  each  group,  which  permits  testing  for  group 
intercept  differences  (Lomax,  1992).  This  assumption  was  supported  by  positive 
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Pearson  correlation  coefficients  (see  Table  4.1).  However,  as  the  table  shows,  the 
Pearson  coefficient  for  treatment  group  3  was  different  than  the  Pearson  coefficients 
for  the  other  three  groups.  To  ensure  compliance  with  this  assumption,  an  analysis  of 
covariance  was  conducted  in  which  a  three-way  interaction  was  specified  between  the 
covariate  (pretest  total  score)  and  the  two  independent  variables  (SPSS  7.5,  1996). 
This  interaction  term,  PROGRAM*SINGLE/PAIRS*PRETEST  TOTAL  SCORE, 
showed  no  evidence  of  violation  of  the  equal  slopes  assumption: 

F(l,  59)=.690,  p=.410.  Thus  the  null  hypothesis  of  equal  slopes  was  accepted. 


Table  4.1 

Pearson  Correlation  Coefficients 


Treatment  Group 

Pearson  Correlation  Coefficient 

Total 

r  =  .490 

Treatment  Group  1  (single/base) 

r  =  .593 

Treatment  Group  2  (single/generative) 

r  =  .568 

Treatment  Group  3  (pair/base) 

r  =  .309 

Treatment  Group  4  (pair/generative) 

r  =  .660 
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Covariate  Measured  Without  Error.  As  discussed  in  Chapter  Three,  the  same 


three  cases  were  used  for  the  pretest  and  posttest.  These  cases  required  the  subjects  to 
evaluate,  synthesize,  and  apply  content  material  while  making  multiple  decisions 
regarding  the  appropriateness  of  lab  test,  assessment,  and  management  actions.  This 
testing  format  precluded  the  ability  to  obtain  a  reliability  estimate  since  the  test  cases 
were  not  objective,  multiple-choice  tests.  However,  these  test  cases  were  developed 
in  conjunction  with  the  practice  cases  and  reviewed  by  transfusion  medicine  experts 
to  establish  content  validity  and  accuracy.  Therefore,  not  meeting  this  assumption  did 
not  adversely  affect  the  results. 

Fixed  Independent  Variables.  This  assumption  states  that  the  researcher  fixes 
the  levels  of  the  independent  variables  (Lomax,  1992).  The  study’s  design,  which 
assigned  subjects  to  one  treatment  group  only,  ensured  compliance  with  this 
assumption. 

Independence  of  the  Co  variate  and  Independent  Variable.  Although  not  an 
assumption  of  the  ANCOVA  model,  the  covariate  should  not  be  influenced  by  the 
independent  variable  (Lomax,  1992).  There  were  two  independent  variables  in  this 
study  -  program  version  and  method  of  instruction.  This  condition  was  met  for 
version  of  program  since  the  pretest  was  taken  before  the  students  entered  the  practice 
portion  of  the  computer  program.  Although  the  students  failed  to  comply  with  the 
request  to  refrain  from  discussing  the  program,  there  appeared  to  be  no  significant 
effect  on  the  pretest  scores.  An  analysis  of  variance  investigating  the  differences  in 
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pretest  scores  over  time  suggests  that  there  were  no  significant  differences  in  pretest 
scores  during  the  course  of  the  study:  F(2,  65)=.217,  p=.  805.  When  asked,  none  of 
the  students  admitted  to  knowing  about  the  content  of  the  pretest  before  running  the 
program.  In  fact,  several  students  commented  that  they  wish  they  had  known  the 
pretest  and  the  posttest  were  the  same. 

Method  of  instruction,  the  second  independent  variable,  was  randomly  assigned 
to  the  unit  teaching  labs  instead  of  each  individual  student.  The  study’s  design 
established  the  “pair”  as  the  unit  of  analysis  (rather  than  two  individuals);  therefore, 
students  assigned  to  the  collaborative  condition  tested  together  rather  than  separately. 
An  analysis  of  variance  conducted  on  the  pretest  scores  demonstrated  no  significant 
difference  between  the  single  and  paired  treatments:  F(l,  64)=.479,  p=.491.  This 
result  suggests  that  method  of  instruction  was  independent  of,  and  did  not  influence, 
the  pretest  scores. 

In  summary,  the  following  assumptions  were  fully  met  for  this  study:  random 
and  independent  errors,  normality,  homogeneity  of  variance,  linearity  and 
homogeneity  of  regression  slopes,  fixed  independent  variables,  and  independence  of 
the  covariate  and  the  independent  variable.  The  assumption  pertaining  to  the 
measurement  of  the  covariate  without  error  could  not  be  tested  due  to  the  nature  of 
the  measure. 
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Summary  and  Descriptive  Statistics 


Table  4.2  presents  summary  and  descriptive  statistics  for  this  investigation.  The 
pretest  scores  ranged  from  13.7  for  the  single/base  treatment  to  15.9  for  the  pair/base 
treatment,  with  an  overall  mean  of  15.1  (SD  =  6.4).  The  pretest  means  ranged  from 
14.8  to  15.5  and  from  14.5  to  15.6  for  the  two  independent  variables  respectively. 

The  overall  mean  for  posttest  performance  was  22.9  (SD  =  5.2).  Posttest  scores 
ranged  from  21.1  for  the  single/generative  treatment  to  24.2  for  the  pair/base 
treatment.  For  the  two  independent  variables,  the  posttest  scores  were  24.1  versus 
21.4  favoring  the  base  version  of  the  program,  and  23.1  compared  with  22.7  favoring 
the  collaborative  condition. 

Table  4.2  also  shows  that  the  mean  time  for  completing  the  practice  portion  of 
the  program  was  72.2  minutes  (SD  =  30.8).  The  mean  completion  time  ranged  from 
63  minutes  to  88.8  minutes.  For  the  two  independent  variables  the  mean  completion 
times  were:  (a)  63.2  minutes  for  the  base  program  versus  84.4  minutes  for  the 
generative  version  of  the  program  and  (b)  70.5  minutes  for  the  pairs  compared  with 
74.1  minutes  for  the  single  students. 
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Table  4.2 


Summary  Statistics 


Program 

Single  or 
Pairs 

N 

Pretest 

Posttest 

Practice 

Case 

Time 

Base 

Single 

18 

M 

13.7 

23.9 

63.4 

SD 

8.6 

Pairs 

20 

M 

15.9 

5.5 

4.9 

Total 

38 

M 

24.1 

k&m 

SD 

7.1 

18.7 

Generative 

Single 

13 

M 

■Egg 

mssm 

88.8 

SD 

5.2 

41.7 

Pairs 

15 

M 

21.6 

80.6 

SD 

5.5 

38.0 

Total 

M 

15.5 

|  21.4 

SD 

5.5 

mm 

Total 

Single 

M 

14.5 

74.1 

7.7 

5.2 

Pairs 

35 

msm 

23.1 

SD 

5.2 

5.2 

Total 

66 

M 

22.9 

SD 

5.2 

— 

Analysis  of  Covariance 

It  was  hypothesized  that  the  embedded  generative  learning  strategies  and 
collaboration  would  lead  to  deeper  processing  of  content  material  and  result  in  higher 
levels  of  learning  as  measured  by  performance  on  the  posttest.  Furthermore,  it  was 
predicted  that  the  two  independent  variables  would  interact  to  influence  learning  and 
posttest  performance.  Analysis  of  covariance  (ANCOVA)  was  selected  to  test  for  the 
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presence  of  these  main  effects  and  interaction.  Table  4.3  presents  the  results  of  the 
ANCOVA.  Table  4.4  presents  the  adjusted  cell  and  marginal  means,  by  treatment 
group. 

Table  4.3 

Analysis  of  Covariance  Summary 


Source 

SS 

df 

MS 

F 

P 

Program 

144.48 

1 

144.48 

7.49* 

.008* 

Singles/Pairs 

0.10 

1 

0.10 

.01 

.943 

Program  X  Singles/Pairs 

6.79 

1 

6.79 

.35 

.555 

Pretest  Total  Score3 

418.83 

1 

418.83 

21.72 

.000 

Error 

1176.40 

61 

19.29 

*  P  <  .05 
a  Covariate 

Note.  The  posttest  total  score  is  the  dependent  variable.  Effect  size  and 
power  at  the  .05  level  are  .61  and  .78  respectively. 


Table  4.4 

Adjusted  Cell  and  Marginal  Means 


Program 

Single  or  Pairs 

Adjusted  Cell 
Means 

Adjusted 

Marginal 

Means 

Base 

Single 

24.20 

Pairs 

Generative 

Single 

21.20 

Pairs 

21.51 

Single 

22.97 

Pairs 

22.89 
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Main  Effect:  Version  of  Program.  The  results  of  the  ANCOVA  suggest  the 


presence  of  a  significant  main  effect  for  the  version  of  program: 

F(l,  61)=7.49,  p=  008.  Inspection  of  the  adjusted  means  indicates  that  subjects 
assigned  to  the  base  version  of  the  program  scored  significantly  higher  on  the  posttest 
than  subjects  assigned  to  the  generative  program. 

Main  Effect:  Method  of  Instruction.  The  preceding  data  suggest  that  there  was 
not  a  significant  main  effect  for  method  of  instruction:  F(l,  61)=.01,  p=.943.  The 
adjusted  means  were  virtually  identical  for  the  single  and  collaborative  treatments. 

Interaction:  Version  of  Program  X  Method  of  Instruction.  The  analysis  of 
covariance  results  suggest  that  the  two  independent  variables  did  not  interact: 

F(l,  61)=.35,  p=.555.  This  indicates  that  the  version  of  program  and  method  of 
instruction  variables  operated  independently  to  affect  learning. 

Summary  of  ANCOVA  Results.  The  preceding  data  suggests  that  there  was  a 
main  effect  for  the  version  of  program  variable,  but  not  in  the  direction  hypothesized. 
Furthermore,  there  was  no  main  effect  for  the  method  of  instruction  variable,  nor  was 
there  an  interaction  between  the  two  independent  variables  as  hypothesized.  Before 
addressing  the  implications  of  these  results,  a  series  of  secondary  analyses  were 
conducted  to  provide  additional  insight  concerning  the  dynamics  of  the  study. 
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Secondary  Data  Analyses 


Multivariate  Analysis  of  Variance 

The  ANCOVA  demonstrated  a  significant  main  effect  for  the  version  of 
program.  Since  the  posttest  consisted  of  three  different  cases,  a  multivariate  analysis 
of  variance  was  conducted  to  determine  where  these  differences  occurred.  Before 
computing  the  MANOVA  it  was  necessary  to  again  test  for  assumptions.  The 
additional  assumptions  include  composite  observations  that  are  normally  distributed, 
equally  variable  in  the  population  sampled,  and  independent. 

Analysis  of  the  histograms  for  the  three  posttest  case  scores  indicates  that  the 
scores  for  test  cases  B  and  C  appear  to  violate  the  assumption  that  composite 
observations  were  normally  distributed.  Comparing  the  ratios  of  the  skewness  and 
kurtosis  statistics  with  the  respective  standard  errors  for  each  test  case  confirmed  the 
violation1.  The  ratios  for  test  case  A  (-1.02  and  -.24  respectively)  were  between  the 
-2  to  +2  parameter  for  normality;  however,  for  test  cases  B  and  C  these  ratios  were 
outside  the  acceptable  range  for  normality  (for  case  B,  skewness  ratio  was  -.2.36;  for 
case  C,  skewness  ratio  was  -4.1  and  kurtosis  ratio  was  3.75).  These  ratios  indicate 
that  the  distributions  for  test  cases  B  and  C  were  more  highly  skewed  to  the  left  than 
normal  and  that  the  tails  for  case  C  were  longer  than  normal.  Levene’s  test  of  equal 
variance,  Cochran’s  C,  and  Bartlett-Box  F  analyses  of  each  posttest  case  suggests  that 
the  assumption  of  equal  variance  was  met  (see  Table  4.5).  Finally,  Box’s  M=T  7.683, 
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F(18, 10791)=.892,  p=.589,  indicates  compliance  with  the  test  for  homogeneity  of 
dispersion  (i.e.,  the  observed  covariance  matrices  of  the  dependent  variables  were 
equal  across  groups).  In  sum,  the  assumptions  of  equal  variability  and  independence 
were  met,  whereas  the  assumption  of  normality  was  not.  However,  multivariate 
analysis  of  variance  is  relatively  robust  to  moderate  departures  from  normality 
(Lomax,  1992),  therefore,  this  violation  had  minimal  effect. 

Table  4.5 

Tests  for  Equal  Cell  Variance 


A 

B 

C 

Levene’s 

Test 

F=1.454, 

p=.24 

F=.147, 

p=.  93 

F=1.695, 

p=.18 

Cochran’s 

C 

C=  .33177, 

p=.57 

C=.33644, 

p=.53 

C=  .40654, 

p=.ll 

Bartlett- 
Box  F 

F=  .84663, 

p=.47 

F=.41605, 

II 

4^ 

F=1.8231, 

p=.14 

The  results  of  the  MANOVA  demonstrate  a  significant  main  effect  for  the 
version  of  program  variable  on  the  posttest:  Wilk’s  F(3,  57)=4.46,  p=.007.  Follow  up 
univariate  tests  found  significant  differences  for  test  case  A  (F(l,  59)=4.17,  p=.046) 
and  test  case  C  (F(l,  59)=4.64,  p=.035)  (see  Table  4.6).  Mean  scores  for  subjects 
using  the  base  program  were  4.5  and  3.6  (for  test  cases  A  and  C  respectively), 
whereas  mean  scores  for  subjects  using  the  generative  program  were  3.0  and  2.9  for 
the  same  test  cases  (see  Table  4.7).  These  results  suggest  that  subjects  using  the  base 
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program  significantly  outscored  subjects  using  the  generative  program  on  these  two 
test  cases. 

Table  4.6 

Univariate  Tests  of  Significance 


Dependent 

Variable 

Sum  of  Squares 

Df 

Mean  Square 

F 

Sig. 

Posttest 
Case  A 

34.88 

1,59 

34.88 

4.17 

.046* 

Posttest 
Case  B 

17.05 

1,59 

17.05 

1.39 

.243 

Posttest 
Case  C 

7.97 

1,59 

7.97 

*  P  <  .05 


Table  4.7 

Posttest  Case  Scores  for  Version  of  Program 


Dependent 

Variable 

Program 

Mean 

Posttest  Case  A 

Base 

4.50 

Generative 

3.01 

Posttest  Case  B 

Base 

16.25 

Generative 

15.21 

Posttest  Case  C 

Base 

3.57 

Generative 

2.86 
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Program  Effectiveness 


The  results  of  the  ANCOVA  suggest  that  subjects  using  the  base  program 
significantly  outperformed  subjects  using  the  generative  version  of  the  program.  The 
results  of  the  MANOVA  indicate  that  the  differences  in  performance  occurred  on  test 
cases  A  and  C  respectively.  This  section  seeks  to  determine  if  the  programs  were 
effective  teaching  principles  of  transfusion  medicine.  To  address  the  question  of 
effectiveness,  a  series  of  one-tailed  t-tests  were  computed  to  (a)  compare  total 
pretest  -  posttest  scores,  (b)  determine  which  components  of  the  test  cases  appeared 
to  stimulate  learning,  and  (c)  specifically  address  the  effectiveness  of  the  generative 
program,  since  students  using  this  version  scored  significantly  lower  on  the  posttest. 

Total  Pretest  -  Posttest  Gain.  The  first  analysis  consisted  of  comparing  the 
pretest  and  posttest  mean  scores  for  the  total  group  of  subjects.  Alpha  was  set  at  .05. 
This  analysis  indicates  that  the  total  posttest  scores  were  significantly  higher  than 
pretest  scores  (t=10.582,  p<.0001). 

Test  Case  Component  Analysis.  The  pretest  and  posttest  were  broken  into  their 
component  parts  to  determine  in  which  portions  of  the  program  posttest  scores  were 
significantly  higher  than  pretest  scores.  This  analysis  consisted  of  running  fifteen 
separate  t-tests,  therefore,  to  control  alpha  from  growing  unacceptably  large,  alpha 
was  set  at  .005  for  each  t-test.  Table  4.8  presents  the  results  of  these  t-tests. 

These  data  suggest  that  posttest  scores  were  significantly  higher  than  pretest 
scores  for  the  three  decision  making  areas  (tLabs=7.817,  p<.005;  tAssess=4.524,  p<.005; 
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and  tManage=5.795,  p<005),  as  well  as  for  all  three  test  cases  (tA=5.34,  p<.005; 
tB=6.97,  p<.005;  tc=4.87,  p<.005).  However,  learning  was  not  uniform  across  the 
three  test  cases.  Lab  scores  were  significantly  higher  on  the  posttest  than  the  pretest 
for  all  three  cases  (tAL-4.814,  p<.005;  tBL=4.408,  p<.005;  and  tcL=3.725,  p<.005), 
whereas  assessment  scores  were  significant  in  the  test  case  B  only  (tBA=4.065, 
p<.005),  and  management  scores  were  significant  in  test  cases  B  and  C  (tBM=2.862, 
p<.005;  tcM=5.744,  p<.005). 
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Table  4.8 


Aggregate  Pretest/Posttest  T-test  Results 


Component 

Mean 

SD 

Df 

T 

Total  Score  - 
Labs 

5.89 

65 

7.817* 

Total  Score  - 
Assessment 

1.09 

1.96 

65 

4.524* 

Total  Score  - 
Management 

1.10 

1.55 

65 

5.795* 

Test  Case  A 
Total 

2.74 

4.17 

65 

5.337* 

Labs 

2.35 

3.96 

65 

4.814* 

Assessment 

.27 

1.651 

Management 

.13 

.72 

65 

1.537 

Test  Case  B 
Total 

3.29 

3.83 

65 

6.974* 

Labs 

2.09 

4.408* 

Assessment 

.82 

1.64 

65 

4.065* 

Management 

.37 

1.03 

65 

2.862* 

Test  Case  C 
Total 

1.83 

3.06 

65 

4.865* 

Labs 

1.23 

2.67 

65 

3.725* 

Assessment 

.00 

.68 

65 

.363 

Management 

.61 

.86 

65 

5.744* 

*  P  < .005 
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Pretest  -  Posttest  Gain  for  the  Generative  Program.  The  final  analysis  compared 
pretest  and  posttest  mean  scores  for  the  students  using  the  generative  version  of  the 
program.  A  one-tailed  t-test  (alpha  =  .05)  indicated  that  posttest  scores  were 
significantly  higher  than  pretest  scores  (t=6.48,  p<.0001).  The  pretest  and  posttest 
were  again  broken  into  their  component  parts  for  analysis.  Alpha  was  set  at  .005. 

The  component  analysis  t-tests  indicate  that  lab  scores  were  significantly  higher 
on  the  posttest  than  the  pretest  (t=4.84,  p<.005),  but  there  were  no  significant 
differences  between  pretest  and  posttest  scores  for  assessment  and  management. 

Also,  the  posttest  scores  were  significantly  higher  than  pretest  scores  for  test  cases  B 
and  C  (tB=3.85,  p<.005;  Tc=2.87,  p<.005).  Finally,  management  scores  were 
significant  for  test  case  C  only  (tcM=3.057,  p<.005). 

In  summary,  total  posttest  scores  for  the  generative  program  were  significantly 
higher  than  pretest  scores;  however  the  difference  appeared  significant  in  only  one 
decision  making  area  (labs),  two  test  cases  (B  and  C),  and  one  part  of  case  C 
(management).  In  contrast,  when  the  analysis  included  both  program  versions, 
posttest  scores  were  significantly  higher  than  pretest  scores  for  all  three  decision 
areas,  all  three  test  cases,  labs  in  all  three  cases,  assessment  in  one  case,  and 
management  in  two  cases.  Thus,  there  appeared  to  be  a  significant  fall  off  in  learning 
effectiveness  for  the  generative  program  when  compared  to  the  aggregate  analysis 
that  included  both  program  versions  (Note:  t-test  analyses  performed  on  the  base 
version  of  the  program  mirror  the  aggregate  results  and  are  presented  in  Appendix  F). 
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Time  Spent  on  Instruction 


Table  4.2  reports  the  amount  of  time  subjects  spent  in  the  practice  portion  of  the 
program.  Analysis  of  variance  (ANOVA)  indicated  a  significant  main  effect  for  the 
version  of  program  (F(l,  62)=8.383,  p=.005),  but  no  main  effect  for  method  of 
instruction  (F(l,  62)=  265,  p=.608).  These  results  indicate  that  subjects  using  the 
generative  version  of  the  program  spent  significantly  more  time  working  on  the  six 
practice  cases  than  those  who  used  the  base  program. 

A  second  ANOVA  was  conducted  to  determine  if  the  time  spent  on  the  practice 
cases  varied  significantly  over  the  course  of  the  study.  Time  spent  in  the  practice 
portion  of  the  program  is  reported  in  Table  4.9.  These  data  reveal  that  there  was  an 
eleven  minute  decline  after  week  one;  however,  an  ANOVA  indicated  that  there  were 
no  significant  differences  in  time  spent  completing  the  practice  cases  over  the 
duration  of  the  study:  F(2, 63)=.874,  p=. 422. 

Table  4.9 

Mean  Time  Spent  on  Instruction 


Week 

Program 

Completed 

Mean  Time  Spent 
On  Practice 
Cases 
(minutes) 

N 

Week  One 

79.8 

20 

Week  Two 

68.7 

■a 

Week  Three 

69.0 

Total 

72.2 

66 
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Analysis  of  Explanations  and  Summaries 

Each  explanation  generated  by  the  subjects  was  coded  relative  to  lesson  content 
as  “irrelevant,”  “partially  relevant,”  or  “relevant.”  If  the  explanation  was  an  incorrect 
synthesis  and  application  of  the  content  material  (i.e.,  obviously  not  related  or  very 
superficially  related  to  the  case)  or  not  attempted,  it  was  classified  as  “irrelevant.” 
When  an  explanation  involved  a  substantially  correct  synthesis  and  application  of  the 
content  material,  it  was  coded  as  “relevant.”  If  an  explanation  involved  synthesis  and 
application  of  information,  but  was  not  substantially  correct,  it  was  categorized  as 
“partially  relevant.”  The  same  scale  was  used  to  classify  the  summaries,  but  relative 
to  the  case  objectives.  Coding  was  accomplished  by  a  physician  with  expertise  in 
transfusion  medicine. 

Table  4.10  reports  the  means  and  standard  deviations  for  the  subjects’ 
explanations  and  summaries.  The  mean  explanation  score  was  24.9  (out  of  a  possible 
total  of  44  points)  and  the  mean  summary  score  was  9.2  (out  of  a  possible  score  of  12 
points).  The  mean  for  total  elaborations  (i.e.,  the  combination  of  explanations  and 
summaries)  was  34.1.  Mean  scores  for  each  elaboration  category  were  divided  by  the 
corresponding  total  possible  points  to  provide  an  indication  of  relevancy  (i.e., 
quality).  This  procedure  produced  relevancy  scores  of  56%  for  explanations,  77% 
for  summaries,  and  61%  for  total  elaborations. 
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Table  4.10 


Summary  Statistics  for  Explanations  and  Summaries 


Singles  or 

Generative 

Generative 

Total 

Pairs 

Explanations 

Summaries 

Singles 

Mean 

23.2 

8.5 

31.8 

N  =  13 

SD 

7.4 

2.4 

9.1 

Pairs 

Mean 

26.3 

9.8 

36.1 

in 

II 

53 

SD 

7.0 

1.6 

8.2 

Total 

Mean 

24.9 

9.2 

34.1 

N  =  28 

SD 

7.2 

2.1 

8.8 

An  ANOVA  was  conducted  to  determine  if  the  mean  differences  between  the 
explanations,  summaries,  and  total  elaborations  generated  by  the  single  students  and 
pairs  were  significant.  Using  an  alpha  of  .05,  there  were  no  significant  differences 
[Fe(1,  26)=1.228,  p=.278;  Fs(l,  26)=2.781,  p=.107;  and  FE+S( 1,26)= 1.721,  p=.201]. 
This  result  suggests  that  collaborating  pairs  were  no  more  successful  at  generating 
explanations  and  summaries  than  the  single  program  users. 

Analysis  of  Student  Comments 

Subjects  were  given  the  opportunity  to  provide  constructive  feedback  after 
finishing  the  program.  These  comments  were  independently  coded  by  two  raters. 
Where  disagreement  occurred,  the  raters  discussed  and  revised  their  codes  to  achieve 
consensus. 

Fourteen  categories  were  initially  established  based  on  the  subjects’  comments. 
These  categories  were  then  collapsed  for  categories  of  responses  that  were  similar. 
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For  example,  some  subjects  directly  critiqued  the  time  required  to  run  the  program, 
while  others  criticized  the  speed  of  the  computers  -  an  indication  of  dissatisfaction 
with  time  (Appendix  G  contains  example  student  comments).  Table  4.1 1  presents  the 
students’  feedback.  The  subjects’  responses  were  negative  relative  to  the  time 
required  to  run  the  program,  as  well  as  the  program  as  an  instructional  method. 
However,  the  subjects’  responses  were  positive  regarding  the  opportunity  and 
benefits  of  collaborative  learning. 

Table  4.11 
Student  Comments 


Category  of  Feedback 

Number  of  Responses 

Time 

Too  much  time 

15 

Computers  too  slow 

8 

Not  enough  time 

2 

Instructional  Method 

Do  not  like  computer-based  instruction 

8 

Prefer  lecture 

11 

Did  not  leam  from  program 

10 

Posttest  cases  differ  from  practice  cases 

15 

Program  is  a  good  educational  tool 

8 

Program  Specific  Feedback 

Lack  of  immediate  feedback  on  test  cases 

14 

Inconvenient  access  to  information  sources 

4 

Inability  to  change  answers 

9 

Collaboration 

Preferred 

13 

Not  preferred 

1 
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Summary  of  Results 


Results  of  the  preceding  statistical  analyses  are  summarized  as  follows: 

1 .  The  ANCOVA  found  a  significant  main  effect  for  the  base  version, 
compared  to  the  generative  version,  of  “Handling  Transfusion  Hazards.” 
There  was  no  significant  difference  in  mean  posttest  scores  by  method  of 
instruction,  nor  was  there  an  interaction  between  the  two  treatment 
variables. 

2.  A  MANOVA  yielded  significant  differences  for  test  cases  A  and  C  for  the 
program  treatment  variable.  Subjects  using  the  base  program 
outperformed  students  using  the  generative  program  on  both  of  these  test 
cases. 

3.  A  series  of  t-tests  found  (a)  total  posttest  scores  significantly  higher  than 
aggregate  pretest  scores  and  (b)  significant  posttest  scores  for  all  three 
decision  making  areas,  all  three  test  cases,  and  most  of  the  component 
parts  of  the  test  cases.  An  identical  analysis  performed  on  the  generative 
version  found  total  posttest  scores  significantly  higher  than  total  pretest 
scores,  but  failed  to  duplicate  the  results  of  the  component  analysis. 

4.  An  ANOVA  on  the  time  data  indicated  that  subjects  using  the  generative 
program  spent  significantly  more  time  in  the  practice  portion  of  the 
program  than  subjects  using  the  base  program.  There  was  no  significant 
difference  in  time  for  the  method  of  instruction  treatment.  Also,  although 
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the  mean  time  declined  after  week  one,  there  was  no  significant  difference 
in  time  spent  on  the  practice  cases  over  the  duration  of  the  study. 

5.  Subjects  using  the  generative  program  provided  “partially  relevant” 
explanations  and  “borderline  relevant”  summaries.  There  was  no 
significant  difference  between  the  quality  of  explanations  and  summaries 
provided  by  the  single  and  paired  students. 

6.  Analysis  of  student  feedback  demonstrated  a  generally  negative 
disposition  toward  the  program  as  an  instructional  method,  as  well  as  the 
time  required  to  complete  the  program.  The  students  were  positive  about 
the  opportunity  and  effects  of  collaboration. 
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CHAPTER  5 


DISCUSSION  AND  RECOMMENDATIONS 

Introduction 

Cognitive  flexibility  theory  (CFT)  has  been  advanced  as  a  means  to  overcome 
the  problem  of  inert  knowledge  formation  by  advanced  learners  in  complex  and  ill- 
structured  knowledge  domains  (Spiro  et  al.,  1988;  1989;  Spiro  &  Jehng,  1990).  The 
research  to  date  supports  the  assertion  that  CFT-based  environments  support  and 
promote  greater  knowledge  acquisition  and  transfer  than  linear  treatments  of  the  same 
subject  content  (Hartman  &  Spiro,  1989;  Jacobson,  1990;  Jacobson  &  Spiro,  1995; 
Spiro  et  al.,  1987).  This  study  accepted  this  premise  and  addressed  the  following 
questions: 

1 .  Would  the  inclusion  of  embedded  explanation  and  summary  cues 
(generative  learning  strategies)  result  in  differences  in  learning  in  a 
CFT-based  computer  microworld? 

2.  How  would  the  addition  of  collaborative  learning  affect  learning  in  this 
environment? 

Based  on  the  literature  reviewed  in  Chapter  Two,  three  hypotheses  were  generated. 
First,  it  was  hypothesized  that  students  assigned  to  the  generative  program  would 
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perform  better  on  the  posttest  than  students  assigned  to  the  base  program.  Second,  it 
was  predicted  that  students  assigned  to  the  collaborative  treatment  would  out  perform 
students  assigned  to  the  individual  condition  on  the  posttest.  Third,  it  was 
hypothesized  that  the  treatment  variables  would  interact  to  increase  learning. 

Discussion 

This  section  relates  the  study’s  findings  to  relevant  research  and  theory.  Each 
hypothesis  is  addressed  relative  to  the  literature  reviewed  in  Chapter  Two. 
Additionally,  other  variables  that  might  account  for  the  study’s  results  are  addressed. 

Effect  of  Embedded  Generative  Learning  Strategies 

Counter  to  the  first  hypothesis,  findings  indicate  that  students  learned  more 
from  the  base  version  of  the  program.  The  analysis  of  covariance  indicated  that 
students  using  the  base  program  significantly  outperformed  students  using  the 
generative  version  on  the  posttest.  The  multivariate  analysis  of  variance  found  that 
the  performance  differences  occurred  on  test  cases  A  and  C.  The  t-test  analyses 
confirm  that  the  base  program  was  superior  to  the  generative  program  in  promoting 
learning.  This  difference  in  posttest  performance  occurred  despite  generative 
program  users  spending  a  significantly  greater  amount  of  time  on  the  practice  cases. 
These  findings  suggest  that  the  embedded  generative  learning  strategies  depressed 


94 


knowledge  acquisition  (relative  to  the  base  program)  by  advanced  learners  in  a 
cognitive  flexibility  theory-based  learning  environment. 

This  finding  deviates  from  studies  showing  the  positive  effects  of  generative 
learning  strategies  on  learning  (Di  Vesta  &  Peverly,  1984;  Johnsey,  Morrison,  & 
Ross,  1992;  Linden  &  Wittrock,  1981;  Slamecka  &  Graf,  1978;  Wittrock  & 
Alesandrini,  1990).  There  are  several  plausible  explanations.  First,  the  discrepancy 
in  results  may  be  due  to  the  context  of  application  of  the  embedded  generative 
strategies.  Explanation  and  summarization  strategies  have  been  used  predominantly 
in  linear,  text-based  environments  that  focus  on  retention  and  recall  of  factual 
information.  In  contrast,  this  cognitive  flexibility  theory-based  microworld,  while 
highly  text-based,  required  the  students  to  synthesize,  evaluate,  and  apply  information 
to  solve  transfusion  medicine  cases.  Thus,  this  study  focused  on  the  effects  of 
generative  learning  strategies  in  a  complex  environment  requiring  critical  thinking 
and  problem  solving.  By  definition,  problem  solving  is  a  generative  activity 
(Wittrock,  1990),  in  which  case,  the  thinking  processes  the  students  used  to  assess 
and  manage  the  cases  in  the  base  program  were  likely  “mindful”  enough  to  support 
knowledge  construction  and  acquisition.  Therefore,  rather  than  complementing  the 
thinking  processes,  the  explanation  and  summary  cues  may  actually  have  been 
redundant. 

Second,  the  subjects  in  this  study  were  high  achieving,  successful  students.  It  is 
likely,  therefore,  that  these  students  could  construct  the  relationships  between  prior 
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knowledge  and  new  information  required  for  comprehension  and  transfer  to  occur 
without  intervention.  Therefore,  asking  these  students  to  provide  explanations  and 
summaries  may  have  been  viewed  by  the  students  as  trivializing  the  learning 
experience  (Wittrock,  1990).  There  is  evidence  to  support  this  argument.  During  the 
course  of  the  study,  one  student  commented  that  she  had  seen  “the  smartest  student  in 
the  class”  running  the  program  and  he  was  frustrated  by  the  necessity  to  type  in 
explanations  and  summaries.  Another  student  typed  in  “What’s  the  point  of  typing  in 
my  responses”  during  the  summary  of  one  of  the  cases.  These  examples  suggest  that 
some  students  viewed  the  necessity  to  type  in  responses  as  a  nuisance,  which  may 
have  influenced  their  interactions  with  the  program  and  their  posttest  performance. 

Third,  the  deficiency  in  learning  for  subjects  using  the  generative  program  may 
relate  to  the  quality  of  explanations  and  summaries  generated.  Elaborations  must  be 
precise  and  relevant  to  facilitate  memory  and  learning  (Di  Vesta,  1989),  yet  the 
quality  of  the  explanations  and  summaries  produced  was  suspect.  The  coding  scheme 
used  to  analyze  the  explanations  and  summaries  suggests  a  tendency  by  the  students 
to  provide  “partially  relevant”  explanations  and  summaries  only  slightly  better.  The 
students  had  not  received  training  on  how  to  generate  effective  explanations  and 
summaries;  therefore,  this  finding  may  indicate  that  the  students  experienced 
difficulty  while  attempting  to  generate  precise  and  relevant  elaborations  and  highlight 
the  need  for  instruction.  Or,  as  discussed  above,  perhaps  the  students  questioned  the 
relevance  and  necessity  to  type  in  elaborated  responses.  In  any  event,  these  results 


96 


support  the  view  that  elaboration  must  meaningfully  relate  new  and  old  information 
to  promote  generative  learning  (Di  Vesta,  1989;  Wittrock,  1974a;  1974b;  1985). 

Fourth,  the  difference  in  posttest  performance  may  be  related  to  the  significantly 
greater  amount  of  time  students  needed  to  complete  the  generative  program.  Table 
4.1 1  shows  that  many  of  the  students  included  in  the  final  analysis  objected  to  the 
amount  of  time  required  to  complete  the  programs.  The  data  suggest  that  some 
students  may  have  believed  the  demands  and  value  of  the  task  were  outside  the 
reasonable  range  of  required  effort  to  learn  the  material  (Salomon  &  Globerson, 
1988).  Consequently,  the  students  may  have  put  less  effort  into  interacting  with  the 
programs.  Since  the  generative  program,  on  average,  took  twenty-one  minutes  longer 
to  complete,  it  is  possible  that  these  students  put  even  less  effort  into  learning  the 
program  content.  The  reduced  effort  may  also  explain  the  average  quality  of 
explanations  and  summaries  produced  by  the  students. 

A  fifth  explanation  may  relate  to  the  structure  of  the  generative  program.  After 
typing  in  explanations  for  lab,  assessment,  and  management  choices,  students  could 
access  expert  feedback  only  by  reselecting  their  choices.  Neither  version  of  the 
program  was  equipped  to  process  an  audit  trail  of  student  activity,  but  the  researcher 
observed  that  some  students  using  the  generative  program  continuously  bypassed  the 
feedback  option.  Bypassing  expert  feedback  may  have  reinforced  the  students’ 
mistakes  and  negatively  influenced  their  posttest  results.  This  explanation,  however, 
deviates  from  studies  that  demonstrate  a  significant  advantage  favoring  subject- 
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generated  elaborations  versus  instruction-provided  elaborations  (Slamecka  &  Graf, 
1978)  or  no  relative  advantage  either  way  (Johnsey  et  al.,  1992). 

Effect  of  Collaboration 

The  second  hypothesis  predicted  that  students  given  the  opportunity  to  collaborate 
would  outperform  students  working  individually  on  the  posttest.  Results  for  posttest 
performance  indicated  no  significant  difference  between  the  single  and  paired 
treatment  conditions.  This  lack  of  significant  advantage  occurred  despite  researcher 
observation  of  some  pairs  that  revealed  a  pattern  of  collaborative  activity  best 
described  as  task  focused  and  co-constructive.  These  students  concentrated  their 
efforts  on  giving  and  receiving  explanations,  which  are  collaborative  behaviors 
positively  correlated  with  increased  achievement  (King,  1989;  Webb,  1987).  Also, 
the  collaborating  students  chose  their  partners,  which  is  thought  to  increase  group 
interdependence  on  short-term  interventions  and  positively  affect  learning  (Salomon 
&  Globerson,  1988).  Even  the  students  themselves  expected  collaboration  to 
increase  learning  and  posttest  performance  (see  Chapter  Four).  For  example,  one 
student  commented,  “I  know  I  would  have  learned  much,  much  more  had  I  been 
working  with  a  partner.”  However,  despite  the  positive  collaborative  interactions, 
the  ability  to  choose  partners,  and  the  students’  expectations,  the  lack  of  significant 
effect  for  method  of  instruction  is  not  surprising  given  the  variable  results  of  studies 
investigating  the  effects  of  computer-based  collaborative  learning  (see  Chapter  Two). 
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There  are  several  possible  explanations  for  the  lack  of  significant  posttest  results 
favoring  collaboration.  These  explanations  may  be  related  to  the  subjects’  academic 
ability,  the  context  of  use,  and  posttest  administration. 

As  medical  students,  these  subjects  were  successful,  high-achieving  learners 
with  similar  academic  ability.  This  lack  of  academic  variability  may  explain  why 
collaboration  did  not  produce  the  differences  in  learning  anticipated.  Four  studies 
comparing  achievement  between  students  learning  individually  and  students  learning 
collaboratively  at  the  same  academic  ability  level  have  produced  mixed  results.  In 
studies  of  elementary  grade  students.  Yager,  Johnson,  and  Johnson  (1985)  found  a 
significant  difference  favoring  collaborating  high-ability  students  relative  to  high 
ability  students  working  individually;  whereas,  Mevarech  (1993)  found  no  significant 
difference  in  achievement  between  high  ability  students  working  independently  or 
collaboratively.  In  a  study  of  college  students,  Gokhale  (1997)  found  that 
collaboration  significantly  enhanced  critical-thinking  and  problem-solving  skills; 
meanwhile,  in  a  study  of  adult  learners  (considered  high  ability),  Makuch,  Robillard, 
and  Yoder  (1992)  found  no  significant  difference  in  achievement  between 
collaborating  pairs  and  individuals.  Although  the  finding  of  no  significant  difference 
was  unanticipated,  this  result  is  not  out  of  line  with  previous  research  and  appears  to 
support  the  findings  of  other  studies  (Hooper  &  Hannafin,  1988;  Makuch,  et  al., 

1992;  Mevarech,  1993)  that  suggest  that  high-ability  students  benefit  the  least 
academically  from  collaborative  learning. 
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The  manner  of  presentation  may  have  also  affected  the  results.  Although 
required  to  complete  each  portion  of  the  program  sequentially  (i.e.,  introduction, 
pretest,  practice  cases,  posttest),  the  students  were  given  considerable  latitude 
regarding  which  resources  they  could  use  while  completing  the  practice  cases.  The 
collaborating  students  spent  less  time  on  the  practice  cases,  which  may  indicate  a 
higher  degree  of  work  efficiency,  but  may  also  indicate  that  the  collaborating  students 
used  less  of  the  available  information  resources  while  working  on  the  practice  cases. 
Since  the  subject  domain  was  new  to  the  students,  this  may  account  for  the  lack  of 
significant  posttest  performance  favoring  the  collaborative  condition. 

The  immediacy  of  the  posttest  may  have  curtailed  the  positive  performance 
effects  often  attributed  to  collaborative  learning.  Since  collaborative  learning  is 
thought  to  promote  deeper  processing  of  material  -  especially  for  problem-solving 
activities  -  by  collaborating  students,  it  is  likely  that  collaborating  students  would 
retain  more  of  the  information  they  learned  (Johnson  &  Johnson,  1996;  Makuch  et  al., 
1992;  Shlechter,  1990;  Yager  et  al.,  1985).  If  this  is  true,  then  an  immediate  posttest 
might  have  masked  treatment  differences  reflecting  deeper  processing  of  material  by 
the  collaborative  condition.  Therefore,  a  delayed  test  designed  to  test  application  of 
retained  knowledge,  rather  than  students’  short-term  retention  and  understanding  of 
content  material,  might  yield  different  results  favoring  the  collaborative  condition.  In 
any  event,  the  finding  of  no  significant  difference  between  individuals  and  pairs  on 
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posttest  performance  supports  Webb’s  (1987)  position  that  group  work  is  not 
detrimental  and  may  be  beneficial  to  students’  learning. 

Interaction  of  Generative  Learning  Strategies  and  Collaboration 

The  third  hypothesis  predicted  that  the  two  treatment  variables  would  interact  to 
affect  performance  on  the  posttest.  The  expectation  was  that  the  interaction  would 
lead  to  greater  performance  by  students  assigned  to  the  pair/generative  treatment.  In 
contrast  to  previous  studies  (Sherman  &  Klein,  1995;  Yager  et  al.,  1985)  that 
demonstrate  the  beneficial  effects  of  cueing  students  to  elaborate  within  a  computer- 
based,  collaborative  learning  environment,  the  results  of  this  study  found  no  such 
benefit  (i.e.,  there  was  no  interaction).  Students  assigned  to  the  pair/generative 
treatment  were  no  more  successful  at  generating  explanations  and  summaries  than 
subjects  assigned  to  the  single/generative  treatment.  Furthermore,  students  assigned 
to  the  pair/generative  treatment  were  outscored  on  the  posttest  by  students  assigned  to 
the  single/base  and  pair/base  conditions. 

The  interaction  effect  was  based  on  the  expectation  that  the  requirement  to 
elaborate  would  increase  the  interaction  between  students,  which  in  turn,  would 
promote  the  generation  of  higher  quality  explanations  and  summaries  favoring  the 
collaborating  students  relative  to  the  students  working  alone.  Direct  observation  of 
eight  pairs  assigned  to  the  generative  program  did  reveal  extra  collaborative 
interactions  when  compared  to  collaborating  students  assigned  to  the  base  program. 
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These  interactions  generally  took  two  forms:  (a)  students  worked  jointly  on  wording 
the  explanations  and  summaries  or  (b)  one  student  typed  while  the  other  student 
corrected  mistakes  or  suggested  additional  information.  Given  the  average  quality  of 
the  explanations  and  summaries  generated  by  the  students,  the  quality  of  these 
interactions  were  most  likely  low  level  information  exchanges  designed  to  satisfy  the 
requirements  of  the  generative  program.  Therefore,  the  lack  of  an  interaction  effect 
may  be  attributable  to  the  inability  of  the  embedded  cues  to  stimulate  effective 
interaction  among  these  students. 

Other  Consideration:  Influence  of  the  Overall  Instructional  Environment 

The  discussion  to  this  point  has  focused  on  explaining  the  findings  relative  to 
each  hypothesis.  Several  explanations  were  discussed  as  possible  reasons  for  the 
results.  It  is  also  likely  that  some  combination  of  these  reasons  may  account  for  the 
study’s  findings.  However,  the  data  suggests  that  a  broader  perspective  may  be 
necessary  to  fully  address  the  research  question.  For  example,  the  t-test  analysis 
suggests  that  “Handling  Transfusion  Hazards”  was  an  effective  learning  environment, 
but  was  it?  The  mean  score  for  the  base  program  was  only  51%  of  the  total  possible 
score  (24  points  out  of  a  total  of  47  points),  and  the  highest  score  obtained  was  32 
points  (68%).  Since  research  has  demonstrated  the  effectiveness  of  cognitive 
flexibility  theory-based  computer  environments  (see  Chapter  Two),  and  given  the 
caliber  of  students  used  in  the  study,  one  might  expect  better  performance  than  an 
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absolute  high  score  of  68%  or  mean  of  51%.  Additional  explanations  may  be  related 
to  the  overall  instructional  environment  of  the  medical  school  curriculum. 

The  first  two  years  of  education  at  this  institution  focus  on  disseminating 
information  to  the  students  and  then  testing  recall  through  multiple-choice  and  true- 
false  examinations  (D.  Ambruso,  personal  communication,  May  13, 1998). 

According  to  Ambruso,  the  curriculum  is  taught  predominantly  in  lectures,  with  little 
opportunity  for  implementation  of  active  learning  strategies.  Also  the  students  are 
highly  extrinsically  (i.e.,  grade)  motivated.  Several  students  corroborated  Ambruso ’s 
statements.  One  student  commented:  “The  medical  school  curriculum  is  mostly 
lecture.  Medical  students  spend  a  great  deal  of  their  time  memorizing  information.” 
Another  student  stated  that  many  of  her  classmates  would  put  minimal  effort  into 
interacting  with  the  program  because  it  was  only  a  pass/fail  exercise,  and  therefore, 
had  virtually  no  impact  on  their  grade  or  class  standing.  For  these  students  the 
traditional  model  of  education  prevails,  punctuated  by  efficient  transmission  of 
information  to  passive  learners  and  multiple-choice  testing  to  evaluate  what  was 
learned  (i.e.,  were  simple  associations  formed  among  the  material?). 

“Handling  Transfusion  Hazards”  imposed  three  demands  on  the  learners  - 
cognitive  complexity,  task  management,  and  buying  in  (Perkins,  1992).  These 
demands  required  the  students  to  think  harder,  think  for  themselves,  and  accept  a  new 
instructional  approach;  yet  many  students’  actions  and  comments  suggest  that  they 
were  accustomed  to  the  traditional  model  of  learning  and  were  surprised  at  the  kind  of 
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learning  required  by  the  program.  For  example,  twenty-six  students  were  dropped 
from  the  final  analysis  due  to  lack  of  effort  when  running  the  program.  Additionally, 
Table  4.1 1,  which  provides  feedback  from  those  students  included  in  the  final 
analysis,  indicates  that  the  students  were  critical  of  (a)  the  time  they  spent  running  the 
program,  (b)  the  case-based  presentation  style  (i.e.,  they  preferred  lecture),  and 
(c)  the  perceived  content  differences  among  the  cases.  These  data  suggest  that  many 
of  the  students  disliked  “Handling  Transfusion  Hazards”  because  it  was  new, 
different,  and  harder  than  the  lecture-based  learning  environment  they  were 
accustomed  to. 

Most  students  have  been  conditioned  by  the  current  educational  system  to 
let  the  “teacher”  organize  their  learning  experiences  for  them.  This  relieves 
the  student  of  the  responsibility  for  thinking  for  themselves.  Thus,  the 
important  learning  tasks  of  planning,  gathering  relevant  resources,  more 
planning,  more  self-directed  learning,  more  decision-making,  and  ultimately 
more  consequences  for  their  own  actions  are  placed  squarely  on  the  students 
and  they  don't  like  it.  Simply  stated,  it’s  harder  work.  (J.  Savery,  personal 
communication.  May  8, 1998) 

The  data  also  suggest  that  since  the  students  were  accustomed  to  the  traditional 
approach  to  learning,  they  would  rather  rely  on  simple  strategies  to  learn  the  material. 
For  example,  one  student  commented:  “Not  that  I  am  old  fashioned,  but  give  me  a 
pencil,  paper,  and  a  book  and  I’m  much  happier.”  Another  student  wrote:  “Your 
practice  cases  were  easier  than  the  pre/post  cases.  Also,  the  practice  cases  do  not 
directly  apply  to  the  test  cases.”  These  comments  suggest  that  the  students  would 
rather  memorize  information  and  rely  on  prototypic  cases  to  learn;  however,  these 
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strategies  result  in  deficient  learning  outcomes  in  complex  and  ill-structured 
knowledge  domains  (Spiro  et  al.,  1988).  Thus,  the  preference  of  some  students  for 
lectures  and  for  memorizing  information  and  relying  on  a  representative  case  to 
instantiate  all  relevant  cases,  may  account  for  the  somewhat  disappointing  learning 
results  and  may  have  affected  the  main  findings  of  this  study.  The  generative 
program  may  have  been  most  affected  because  it  required  more  mental  effort. 

A  second  explanation  may  involve  the  inability  of  the  students  to  deeply  process 
the  subject  content.  As  previously  discussed,  the  students  had  access  to  case  specific 
history  and  physical  exam  information,  perspectives  of  case-relevant  operatives,  a 
bank  of  similar  cases,  and  the  transfusion  medicine  textbook.  Although  not  directly 
measured,  the  researcher  noted  a  relationship  between  the  amount  of  time  spent  in  the 
practice  cases  and  the  amount  of  information  accessed.  This  increased  time  would 
then  denote  information  searches  necessary  for  the  students  to  assess  and  manage  the 
clinical  problems  presented  in  the  cases.  Since  “Handling  Transfusion  Hazards”  was 
information  rich,  then  it  would  be  reasonable  to  expect  a  positive  correlation  to  exist 
between  time  and  posttest  performance.  While  not  reported  in  Chapter  Four,  a 
correlation  was  computed  and  indicated  a  slightly  negative,  but  not  significant, 
relationship  (r  =  -.  1 85)  between  the  time  spent  on  practice  cases  and  posttest  scores. 
This  suggests  that  some  students  may  have  suffered  from  cognitive  overload  (mental 
fatigue)  due  to  the  number  of  information  options  available.  Since  students  using  the 
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generative  program  spent,  on  average,  twenty-one  minutes  longer  interacting  with  the 
generative  program,  perhaps  they  suffered  most  from  cognitive  overload. 

Summary 

Would  the  integration  of  compatible  generative  learning  activities  into  a 
cognitive  flexibility  theory-based  learning  environment  promote  deeper  processing  of 
material,  and  consequently,  increased  knowledge  acquisition  by  advanced  learners? 
The  findings  of  this  study  suggest  that  the  answer  is  no.  Rather  than  stimulate  the 
mindful  processes  of  successful  learning,  the  results  suggest  that  embedded 
explanation  and  summary  cues  (generative  learning  strategies)  had  the  opposite  effect 
relative  to  the  control  condition  (i.e.,  base  program).  Students  using  the  base  program 
outperformed  students  using  the  generative  program  on  the  posttest.  Additionally, 
despite  the  apparent  effective  behavioral  involvement  and  student  expectations, 
collaborative  activity  had  no  significant  effect  on  posttest  performance.  This  suggests 
that  there  is  no  relative  advantage  or  disadvantage  to  computer-based  collaborative 
learning  for  advanced  learners.  Finally,  since  the  embedded  generative  strategies 
depressed  knowledge  acquisition  (relative  to  the  control  condition)  and  there  was  no 
advantage  to  working  collaboratively,  the  expected  interaction  failed  to  materialize. 

Recommendations  for  Future  Research 

The  research  question  that  framed  this  investigation  addressed  whether 
embedded  generative  strategies  and  collaboration  would  benefit  advanced  learners  in 
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a  CFT-based  learning  environment.  The  findings  suggest  that  the  answer  is  no,  but 
many  questions  remain.  For  example,  would  generative  learning  strategies  benefit 
advanced  learners  in  a  different  learning  environment?  Was  the  negative  effect  for 
generative  learning  strategies  due  solely  to  the  quality  of  the  students  used  in  the 
study  (i.e.,  advanced  learners),  use  in  a  problem-solving  environment,  or  both?  In 
addition,  the  findings  support  the  relationship  between  the  quality  of  the  generated 
elaborations  and  learning  (Di  Vesta,  1989),  but  contradict  the  relationship  between 
learner-generated  elaborations  and  learning  relative  to  instruction-provided 
elaborations  (Johnsey  et  al.,  1992;  Slamecka  &  Graf,  1978).  Given  the  importance  of 
facilitating  the  mindful  processes  of  learning,  more  research  devoted  to  the  contextual 
application  of  generative  learning  strategies  is  necessary. 

With  respect  to  collaborative  learning,  was  the  lack  of  significant  difference 
between  individual  students  and  pairs  due  to  the  quality  of  student,  the  learning 
environment,  or  both?  The  students  were  permitted  to  select  their  partners,  what 
effect  did  this  have  on  collaborative  activity  and  achievement?  Research  suggests 
that  simply  instructing  students  to  work  together  is  ineffective  (Carrier  &  Sales, 

1987),  but  some  students  exhibited  the  constructive  intra-group  interactions  and 
behaviors  associated  with  successful  group  work  (Damon  &  Phelps,  1989;  King, 
1989;  Webb,  1987).  This  study  was  not  designed  to  directly  evaluate  student 
interactions,  but  since  some  students  (and  perhaps  most  or  all)  collaborated 
successfully,  why  the  finding  of  no  difference  in  performance?  This  suggests  that 
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future  research  should  focus  on  the  quality  of  the  interactions  (i.e.,  the  conversational 
content),  not  the  quantity  of  specific  behaviors. 

Cognitive  flexibility  theory  evolved  from  the  observed  learning  deficiencies  of 
medical  school  students  (Spiro  et  al.,  1988),  yet  most  of  the  relevant  research  has 
used  volunteer  advanced  high  school  and  college  students  to  validate  the  efficacy  of 
the  theory.  The  present  study  used  second-year  medical  students  who  were  required 
to  complete  a  computer  program  developed  according  to  CFT  principles.  This 
investigation  did  not  directly  assess  the  effectiveness  of  cognitive  flexibility  theory, 
but  the  data  and  results  suggest  more  research  is  warranted.  For  example,  the  results 
suggest  that  “Handling  Transfusion  Medicine”  promoted  learning,  but  just  how 
effective  was  it?  Despite  providing  feedback  and  an  organizational  structure  to  help 
students  manage  the  complexity  of  information,  many  students  appeared  to  have 
difficulty  processing  the  information  contained  in  the  program.  Was  this  due  to  the 
problem-solving  context  of  application?  Is  additional  scaffolding  or  coaching  help 
necessary  for  advanced  learners  to  function  highly  in  a  CFT-based  environment? 

Finally,  a  larger  question  must  be  answered  regarding  the  integration  of 
cognitive  flexibility  theory  or  any  other  active,  student-centered  learning 
environment.  The  data  suggest  that  many  of  the  students  simply  did  not  put  much 
time  or  effort  into  interacting  with  the  computer  program  used  in  this  study.  These 
students  were  accustomed  to  memorizing  and  reproducing  information  and  therefore, 
unwilling  or  unable  to  deeply  process  the  information.  Thus,  this  study  points  to  the 
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need  to  address  the  question  of  what  circumstances  or  conditions  are  necessary  to 
successfully  implement  an  active,  student-centered  learning  environment. 

Study  Limitations 

The  research  reported  in  this  study  was  complicated  by  the  problems  associated 
with  conducting  research  at  a  medical  school;  specifically,  the  necessity  to  provide  a 
fair  degree  of  flexibility  to  accommodate  the  students’  schedules.  The  students  were 
required  to  complete  the  program  as  part  of  the  hematology  rotation  of  the 
Pathophysiology  of  Disease  course;  however,  no  time  was  allocated  for  this  purpose 
during  the  students’  academic  day.  The  students  had  to  balance  competing  demands 
from  other  courses,  and  given  the  pass/fail  grade  designation  for  this  exercise  relative 
to  the  requirements  of  the  other  courses,  some  students  chose  to  minimize  how  they 
interacted  with  the  program.  Additionally,  only  three  computers  were  available  for 
the  students  to  use.  Therefore,  to  provide  the  flexibility  required,  the  students  were 
given  approximately  three  weeks  (March  23,  1998  to  April  10, 1998)  to  complete  the 
program. 

The  students  were  asked  to  complete  the  program  as  assigned  (i.e.,  individually 
or  collaboratively)  and  to  refrain  from  discussing  program  content  or  operation  with 
one  another  until  after  the  hematology  course  ended.  There  was  no  practical  way  to 
enforce  this  request;  however,  to  monitor  the  level  of  compliance,  the  researcher 
observed  a  large  number  (84)  of  the  subjects  running  the  program  and  asked  many 
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students  what  they  had  heard  about  the  program  from  other  students.  Only  two 
subjects  were  observed  violating  the  requirement  to  work  as  assigned,  which  provided 
reasonable  assurance  that  the  students  were  abiding  by  this  condition.  However, 
every  subject  the  researcher  questioned,  reported  hearing  something  about  the 
program  from  other  students.  The  typical  comments  were  that  the  program  was 
“long”  or  “frustrating.”  These  types  of  comments  may  have  affected  the  students’ 
attitudes  and  motivation  toward  the  program  and  altered  their  interaction.  The 
generative  version  of  the  program  may  have  been  especially  susceptible  to  these 
influences. 

The  final  limitation  involved  the  pretest-intervention-posttest  format  of  the 
computer  program.  Since  the  program  was  developed  to  provide  a  comprehensive 
learning  environment  providing  both  content  instruction  and  evaluation,  the  pretest 
and  posttest  were  separated  only  by  the  time  required  to  finish  the  six  practice  cases 
(intervention).  Also,  since  the  program  was  developed  to  provide  a  complete  learning 
experience,  it  was  deemed  inappropriate  to  appreciably  change  the  program  structure, 
except  to  develop  a  second  generative  version.  Therefore,  the  same  cases  were  used 
for  both  the  pretest  and  the  posttest  (see  Chapter  Three).  Although  not  ideal  due  to 
the  potential  testing  effect,  it  would  have  been  time  and  cost  prohibitive  to  develop 
the  requisite  number  of  different  cases  for  the  pretest  and  posttest  and  still  maintain 
the  same  level  of  experimental  rigor.  Also,  given  the  logistics  of  the  study,  it  would 
have  been  extremely  difficult  to  use  a  delayed  posttest.  Thus,  while  not  eliminated, 
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the  pretest-posttest  control  group  design  reduced  the  effect  of  testing  by  subjecting  all 
students  to  the  same  testing  format.  Analysis  of  variance  testing  indicates  that  there 
was  little  variation  in  pretest  and  posttest  scores  over  the  course  of  the  data  collection 
period.  Therefore,  the  posttest  scores  of  all  the  subjects  were  most  likely  similarly 
influenced  by  the  pretest. 
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PNebotomist 


PRACTICE  CASES:  Perspective  of  Blood  Bank  Director 


The  additional  information  you  and  the  phlebotomist  have 
gathered  is  very  useful. 

The  usual  volume  of  blood  collected  is  approximately  500 
ml.  This  Is  relative  to  a  total  blood  volume  ranging  from 
3,500  to  7,000  ml,  depending  on  the  size  and  weight  of  the 
donor.  Most  donors  tolerate  giving  blood  very  well,  but 
occasionally  adverse  reactions  may  occur. 

It  sounds  like  this  donor  may  have  had  psychologic,  as 
well  as  neurophysiologic,  reasons  for  this  syncopal  episode, 
as  he  expressed  nervousness  about  donating  and  also  may 
be  relatively  dehydrated  due  to  die  comb  (nation  of  caffeine 
and  fasting. 


PRACTICE  CASES:  Perspective  of  Blood  Bank  Director  cont. 


The  donor  should  have  been  temporarily  deferred  until  he 
had  received  a  meal  orsnack  because  we  routinely  require 
that  the  patient  has  eaten  in  the  previous  foir  hours.  The 
clinical  picture  is  very  characteristic  of  a  vasovagal  attack. 
Occasionally  donors  who  actually  lose  consciousness  will 
have  brief,  convulsion-like  movements  in  the  exlremities. 

I  am  glad  you  are  available  to  provide  medical  coverage 
and  I  hope  this  has  been  an  educational  experience.  For 
further  information  on  the  incidence  of  vasovagal  reactions 
and  preventive  measures  reviewthe  appropriate  section  in 
the  textbook. 


SIMILAR  CASES  MENU 
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PRACTICE  CASES:  Management 


APPENDIX  B 


SAMPLE  GENERATIVE  SCREENS 


The  Introduction,  History/Physical,  and  Perspectives  segments  are  identical  in  the  basic  and  generative 
versions.  The  following  screens  are  different  for  the  generative  version. 
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SAMPLE  TEST  CASE 
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[Related  to  the  fart  that  Wood  given  wen  not  type-specfflcforthe  patient. 


TEST  CASE  B:  Summary 


Congratulations!  You  have  competed 
The  Case  of  the  Dilutional  Dilemma 

This  test  measured  your  ability  to: 

•  Define  the  risk  of  hemolytic  transfusion  reaction 

•  Identify  the  signs  and  symptoms  of  a  hemolytic 
transfusion  reaction 


Investigate  the  causes  of  hemolytic  transfusion 
reactions  including  non-lmmune  hemolysis 

Apply  the  principles  for  managing  a  hemolytic 
transfusion  reaction 


APPENDIX  D 


STUDENT  INSTRUCTIONAL  HANDOUT 


MEMO 

To:  Medicine  6000  (Pathophysiology)  Students 

From:  _ ,  MD 

Date:  Monday,  March  23, 1998 

Subject:  Transfusion  Medicine  Computer-Based  Learning  Program 

As  students  enrolled  in  Medicine  6000,  you  will  complete  “Handling  Transfusion 
Hazards,”  a  computer-based  instruction  program  that  provides  you  with  a  unique 
opportunity  to  learn  about  the  risks  associated  with  blood  transfusions.  The  program 
consists  of  several  authentic  clinical  cases  that  cover  a  broad  array  of  transfusion 
medicine  topics  for  donors  (e.g.,  pain,  infection,  bleeding  at  the  phlebotomy  site,  and 
vasovagal  reactions)  and  recipients  (e.g.,  transfusion  transmitted  viruses,  immediate 
and  delayed  hemolytic  reactions,  and  non-immune  adverse  events  of  transfusions). 
“Handling  Transfusion  Hazards”  requires  you  to  collect  relevant  information  from 
several  on-line  resources  to  help  you  order  and  interpret  lab  tests,  assess,  and 
manage  the  transfusion  problems  presented.  This  program  simulates  the  thinking 
and  problem-solving  processes  used  by  practicing  clinicians.  Transfusion  medicine 
is  a  burgeoning  field  in  medicine;  this  may  be  your  only  formal  exposure  to  this 
information  in  medical  school. 

Procedures 

1.  The  program  is  loaded  on  three  Macintosh  computers  located  in  the  Learning 
Resources  Center  (LRC)  on  the  3rd  floor  of  the  Dennison  Library.  These  computers 
are  prominently  labeled  with  yellow  placards.  You  will  be  given  a  diskette  to 
access  the  program;  the  diskette  will  record  your  interactions  with  the  program  and 
must  be  turned  into  the  LRC  staff  when  you  finish  the  program.  This  is  a  Pass/Fail 
exercise  that  you  must  complete  by  April  13, 1998.  If  you  do  not  complete  the 
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program  by  this  date,  a  “failure”  notation  (for  this  exercise)  will  be  entered  into 
your  record  for  the  hematology  rotation  of  Medicine  6000.  Please  note:  the 
hematology  final  exam,  scheduled  for  April  13, 1998,  will  contain  transfusion 
medicine  questions  derived  from  the  program.  It  is  advisable,  therefore,  to  finish 
the  program  by  Friday,  April  10. 

2.  To  help  us  evaluate  the  effectiveness  of  this  program,  you  will  complete  the 
program  either  individually  or  in  pairs.  Students  in  the  following  UTLs  will  work 
individually:  Room  1862B,  Room  2809  (Inner),  and  Room  1860.  All  other 
students  will  work  collaboratively  with  a  partner  of  your  choosing  from  your  UTL. 

To  maintain  the  integrity  of  the  program,  complete  the  program  according  to  this 
condition  (i.e.,  by  yourself  or  collaboratively  with  your  chosen  partner)  and 
refrain  from  discussing  program  content  or  operation  with  any  other  Med  6000 
students. 

3.  Given  the  large  number  of  students  and  the  limited  resources  (number  of  computers 
and  time)  available,  we  have  developed  a  schedule  to  alleviate  potential  frustration 
and  to  ensure  the  timely  completion  of  the  program.  The  schedule  begins  on 
Wednesday,  March  25  and  runs  through  April  9.  The  schedule  allocates  computer 
time  in  three  hour  blocks  on  a  Monday,  Wednesday,  Friday,  and  Saturday  format. 
Please  note,  the  computers  are  available  for  use  on  Tuesday,  Thursday,  and  Sunday 
on  a  first-come,  first-served  basis,  but  you  run  the  risk  of  not  gaining  access  to  a 
computer  right  away  on  these  days.  The  schedule  is  available  for  sign  up 
immediately  following  the  initial  lecture  (until  1  p.m.)  and  then  will  be  available  in 
room  1862A  until  5  p.m.  Wednesday,  March  25.  Copies  of  the  schedule  will  be 
posted  in  each  UTL  and  next  to  the  computers  in  the  LRC.  Please  respect  the 
schedule  and  adhere  to  your  time. 

4.  Disks  are  available  for  pick  up  immediately  following  the  first  lecture  (until  1  p.m.) 
and  then  will  available  in  your  UTL.  Please  complete  the  information  requested  on 
the  sign-up  sheet  when  you  pick  up  your  disk.  Also  print  your  name  and  student 
number  on  your  disk.  For  students  working  collaboratively,  please  be  sure  to  sign 
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for  the  same  disk  as  your  partner,  and  print  both  names  and  student  numbers  on  the 
disk. 


Running  the  Program 

5.  The  program  should  take  approximately  2  to  2  1/2  hours  to  complete.  You  do 
not  have  to  complete  the  program  in  one  sitting,  although  it  is  advisable  to  do  so. 

If  you  choose  to  complete  the  program  in  two  or  more  sittings,  please  ensure  that 
you  pick  a  time  that  does  not  conflict  with  anyone  else. 

6.  The  program  consists  of  an  orientation  section,  pretest,  learning  phase,  and 
posttest.  The  orientation  discusses  the  learning  objectives  and  provides  valuable 
information  about  the  other  sections  of  the  program— take  the  time  to  read  through 
this  section.  The  pretest  consists  of  three  cases,  the  learning  phase  contains  six 
cases,  and  the  posttest  includes  three  cases.  The  pretest  provides  a  baseline  of 
your  knowledge  and  will  be  used  to  judge  the  effectiveness  of  the  program  through 
comparison  with  the  posttest.  The  on-line  resources  available  during  the  learning 
portion  of  the  program  are  not  available  during  the  pretest  or  posttest.  Your  pretest 
and  posttest  scores  will  be  available  for  viewing  after  completing  the  three  posttest 
cases.  Analyses  of  the  pretest  and  posttest  cases  will  be  available  for  pick  up  on 
Friday,  April  10,  in  the  LRC. 

7.  A  researcher  may  ask  to  observe  you  work-please  oblige  him.  He  will  not 
interfere  with  you,  but  will  record  observational  data,  such  as  program  navigation 
patterns,  collaborative  behavior  patterns,  etc.  This  information,  and  pre-  and  posttest 
comparisons,  will  help  us  modify  the  program  to  optimize  its  utility  as  a  learning  exercise. 

8.  The  program  prompts  you  to  enter  your  name  and  identification  number.  Paired  students 
should  enter  both  names  and  identification  numbers. 

9.  If  the  unexpected  happens  and  you  encounter  technical  problems,  alert  the  LRC 
staff  and  the  on-site  researcher.  If  the  problem  cannot  be  fixed  by  these  individuals, 

exit  the  program  and  contact _ (363-2235)  or _ (363-2241) 

who  will  arrange  to  correct  the  problem. 
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APPENDIX  E 


PRETEST/POSTTEST  DATA  SUMMARY 


Subject 

Treatment  Group 

Pretest 

Posttest 

1 

Pair/Generative 

19 

32 

2 

Pair/Base 

19 

29 

3 

Pair/Generative 

20 

27 

4 

Pair/Base 

0 

22 

5 

Pair/Generative 

10 

20 

6 

Pair/Base 

18 

25 

8 

Single/Base 

9 

14 

9 

Single/Generative 

15 

25 

12 

Single/Base 

-5 

19 

13 

Single/Generative 

17 

20 

15 

Single/Generative 

10 

27 

16 

Single/Base 

18 

26 

17 

Single/Generative 

6 

13 

18 

Single/Base 

17 

16 

19 

Pair/Generative 

24 

26 

20 

Pair/Base 

21 

32 

21 

Pair/Generative 

13 

26 

22 

Pair/Base 

13 

26 

24 

Pair/Base 

18 

21 

25 

Pair/Generative 

8 

19 

26 

Pair/Base 

9 

18 

27 

Pair/Generative 

15 

21 

28 

Single/Base 

3 

25 

30 

Pair/Base 

25 

25 

32 

Single/Base 

10 

22 

33 

Single/Generative 

30 

29 

34 

Single/Base 

12 

22 

37 

Single/Generative 

15 

13 

38 

Single/Base 

11 

18 

40 

Single/Base 

8 

22 

41 

Single/Generative 

14 

20 
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Subject 

Treatment  Group 

Pretest 

Posttest 

42 

Pair/Base 

18 

23 

43 

Pair/Generative 

11 

18 

44 

Pair/Base 

11 

32 

45 

Pair/Generative 

8 

24 

47 

Pair/Generative 

7 

10 

48 

Pair/Base 

16 

21 

49 

Pair/Generative 

14 

21 

50 

Pair/Base 

17 

21 

52 

Pair/Base 

19 

29 

54 

Pair/Base 

9 

13 

56 

Pair/Base 

18 

21 

58 

Pair/Base 

19 

26 

59 

Pair/Generative 

18 

17 

61 

Single/Generative 

8 

16 

62 

Single/Base 

13 

29 

63 

Single/  Generative 

20 

24 

64 

Single/Base 

21 

31 

66 

Single/Base 

22 

27 

67 

Single/Generative 

11 

21 

68 

Single/Base 

33 

31 

69 

Single/Generative 

15 

18 

70 

Single/Base 

25 

29 

71 

Single/Generative 

20 

27 

72 

Single/Base 

14 

28 

74 

Single/Base 

14 

26 

75 

Single/Generative 

23 

21 

76 

Single/Base 

6 

24 

78 

Single/Base 

15 

25 

79 

Pair/Generative 

21 

27 

80 

Pair/Base 

14 

27 

81 

Pair/Generative 

16 

17 

82 

Pair/Base 

20 

18 

83 

Pair/Generative 

16 

18 

84 

Pair/Base 

13 

26 

88 

Pair/Base 

20 

29 
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Histogram  of  Posttest  Scores 


16 


12.5  17.5  22.5  27.5  32.5 


Posttest  Total  Score 


Stem  and  Leaf  Plot  Analysis 


Posttest  Total 

Score 

Stem-and-Leaf  Plot 

Frequency 

Stem  & 

Leaf 

5.00 

1  . 

03334 

12.00 

1  . 

667788888999 

20.00 

2  . 

00011111111222223444 

24.00 

2  . 

555566666667777778999999 

5.00 

3  . 

11222 

Stem  width:  10.00 
Each  leaf:  1  case(s) 


138 


Correlation  Graphs 


Posttest  Total  Score 


Posttest  Total  Score  *  Single/Base  Treatment 


Posttest  Total  Score  -  Single/Generative  Treatment 
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Pretest  Total  Score  -  Pair/Generative  Treatment  Pretest  Total  Score  -  Pair/Base  T reatment 


30 1 


Posttest  Total  Score  -  Pair/Base  Treatment 


Posttest  Total  Score  -  Pair/Generative  Treatment 
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Component  Analysis:  Generative  and  Base  Programs 


Base 

Generative 

Total  Score  - 
Labs 

6.210* 

27 

4.841* 

Total  Score  - 
Assessment 

37 

4.124* 

27 

2.097 

Total  Score  - 
Management 

37 

7.164* 

27 

1.880 

Test  Case  A 
Total 

Q 

4.864* 

27 

2.515 

Labs 

4.396* 

27 

2.235 

Assessment 

37 

1.859 

27 

.171 

Management 

37 

1.743 

27 

.465 

Test  Case  B 
Total 

37 

5.875* 

27 

3.849* 

Labs 

37 

3.553* 

27 

2.580 

Assessment 

37 

3.019* 

27 

2.725 

Management 

37 

3.064* 

27 

.779 

Test  Case  C 
Total 

37 

3.903* 

27 

2.869* 

Labs 

37 

2.543 

27 

2.733 

Assessment 

37 

-1.072 

Management 

37 

3.057* 

*  P  <  .005 


141 


APPENDIX  G 


SAMPLE  STUDENT  COMMENTS 


Time 

Too  much  time 

1 .  Too  time  consuming  to  be  incorporated  into  the  rest  of  the  medical  school 
schedule  with  too  low  a  return  of  knowledge  per  time  spent. 

2.  Too  much  time  involved. 

Computers  too  slow 

1 .  I  found  this  program  exceptionally  slow. 

2.  These  computers  are  way  to  slow  to  make  this  program  effective. 

Not  enough  time 

1 .  I  probably  would  have  spent  more  time  and  learned  a  little  more  if  I  didn’t 
have  a  test  tomorrow. 

2.  The  program  could  be  good  if  we  had  more  time  to  spend  on  it. 

Instructional  Method 

Do  not  like  computer-based  instruction 

1 .  Questions  are  not  possible  using  this  method  of  instruction. 

2.  Rob  does  not  like  computer-based  learning. 

Prefer  lecture 

1 .  Our  low  scores  on  the  post  test  were  due  to  not  learning  very  specific 
details  which  in  my  opinion  would  be  better  learned  from  a  handout  or 
book. 

2.  I  think  the  material  could  be  learned  in  less  time  with  simple  text  learning. 

Did  not  learn  from  program 

1 .  Our  evaluation  of  the  program  reflects  our  disbelief  that  our  answers  were 
incorrect.  If  this  is  true,  we  did  not  leam  a  damn  thing. 

2.  Disappointingly,  our  posttest  scores  were  barely  better  than  our  pretest. 

Posttest  cases  differ  from  practice  cases 

1 .  The  learning  cases  did  not  seem  to  correlate  with  the  test  cases  as  the  test 
was  HCV  and  the  learning  was  HBV.  We  were  just  as  lost  on  posttest 
cases  1  &  2  as  we  were  before  we  began  the  learning. 
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2.  It  would  have  been  beneficial  to  have  the  cases  in  the  pre  and  posttests  in 
the  learning  exercises,  so  we  can  definitely  improve. 

Program  is  a  good  educational  tool 

1 .  This  program  was  effective  in  making  me  think  about  how  to  order  tests 
and  learn  information  about  patients. .  ..This  was  an  excellent  exercise  to 
practice  clinical-based  medicine  for  next  year. 

2.  It  was  a  pretty  good  program  for  integrating  knowledge  of  lab  tests  and 
physiologic  conditions  which  we  have  studied.  I  thought  it  did  a  pretty 
good  job  of  helping  me  to  think  through  decisions  and  the  rationale  behind 
different  lab  tests/interventions. . ..I  liked  how  it  concentrated  on  decision 
making. 

Program  Specific  Feedback 

Lack  of  immediate  feedback  on  test  cases 

1 .  We  felt  extremely  discouraged  by  the  fact  that  there  was  no  feedback  on 
the  answers  we  got  wrong  (which  were  numerous)  -  what  were  the  correct 
answers? 

2.  It  would  have  been  nice  to  have  explanations  for  the  answers  on  the 
exams. 

Inconvenient  access  to  information  sources 

1 .  Information  in  this  program  is  way  too  inconvenient  to  access. 

2.  It  was  too  time  consuming  and  difficult  to  find  the  information  I  needed. 

Inability  to  change  answers 

1 .  This  program  must  have  the  ability  to  take  back  answers. 

2.  I  would  like  to  be  able  to  “unclick”  an  answer  if  I  change  my  mind. 

Collaboration 

Preferred 

1 .  Please  make  the  definitive  decision  that  people  work  better  and  learn  more 
in  pairs. . .1  know  I  would  have  learned  much,  much  more  had  I  been 
working  with  a  partner. 

2.  We  definitely  enjoyed  the  group  learning  experience  and  found  it  very 
helpful. 

Not  preferred 

1 .  It  would  be  better  not  to  try  to  force  people  to  do  it  in  pairs. 
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